2024-04-25T13:01:17.859 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T13:01:17.859 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465 --name leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} -- /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465/orig.config.yaml 2024-04-25T13:01:17.919 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:01:18.004 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} email: null first_in_suite: false job_id: '7673465' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 4d7908725372ec8e17504b848bfa122dd1d43386 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4060 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 4d7908725372ec8e17504b848bfa122dd1d43386 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBENxculZ/TityTJ5EObalVdLeOU1lw0PUsQ4uJQS0xZgVWQ/NbSSG2Unzb3VdtTAauyty/I7WY1BIxfTdzEtP/Q= smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIgCIB34l20qAyUdpOx7bVIvJKOsg8rPP8WrxCRIbM+XOXl6uNMhgF5UR4qkSY12zGH1s5GKDR0Zyf5DgWuwHM= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKJxYTDXhsmnnsvr4HCEHoYAQZDi3Gu03CCr10KeuA/HLJ9vpoMpowtuVVY3cC2gh06LA4fTkKS+v5EfIAK8XMM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_12:17:05 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T13:01:18.004 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa; will attempt to use it 2024-04-25T13:01:18.005 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks 2024-04-25T13:01:18.005 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T13:01:18.007 INFO:teuthology.task.internal:Checking packages... 2024-04-25T13:01:18.030 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T13:01:18.030 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T13:01:18.030 INFO:teuthology.packaging:ref: None 2024-04-25T13:01:18.030 INFO:teuthology.packaging:tag: None 2024-04-25T13:01:18.030 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T13:01:18.030 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:01:18.030 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T13:01:18.197 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T13:01:18.199 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T13:01:18.206 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T13:01:18.206 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T13:01:18.218 INFO:teuthology.task.internal:Saving configuration 2024-04-25T13:01:18.239 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T13:01:18.245 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:01:18.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:55:27.200103', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBENxculZ/TityTJ5EObalVdLeOU1lw0PUsQ4uJQS0xZgVWQ/NbSSG2Unzb3VdtTAauyty/I7WY1BIxfTdzEtP/Q='} 2024-04-25T13:01:18.297 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:55:27.201363', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIgCIB34l20qAyUdpOx7bVIvJKOsg8rPP8WrxCRIbM+XOXl6uNMhgF5UR4qkSY12zGH1s5GKDR0Zyf5DgWuwHM='} 2024-04-25T13:01:18.319 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:55:27.202430', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKJxYTDXhsmnnsvr4HCEHoYAQZDi3Gu03CCr10KeuA/HLJ9vpoMpowtuVVY3cC2gh06LA4fTkKS+v5EfIAK8XMM='} 2024-04-25T13:01:18.320 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T13:01:18.326 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T13:01:18.326 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T13:01:18.326 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T13:01:18.326 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T13:01:18.425 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1fe0e5d0d0>, signals=[15]) 2024-04-25T13:01:18.425 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T13:01:18.431 INFO:teuthology.task.internal:Opening connections... 2024-04-25T13:01:18.431 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T13:01:18.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:01:18.513 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-04-25T13:01:18.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:01:18.589 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-04-25T13:01:18.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:01:18.667 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T13:01:18.673 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-04-25T13:01:18.692 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-04-25T13:01:18.693 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-04-25T13:01:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-04-25T13:01:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-04-25T13:01:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-04-25T13:01:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:01:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-04-25T13:01:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:01:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:01:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-04-25T13:01:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:01:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:01:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:01:18.752 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:01:18.752 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:01:18.752 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:01:18.752 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-04-25T13:01:18.772 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-04-25T13:01:18.791 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-04-25T13:01:18.791 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-04-25T13:01:18.849 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream" 2024-04-25T13:01:18.849 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="9" 2024-04-25T13:01:18.849 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos" 2024-04-25T13:01:18.849 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:01:18.849 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="9" 2024-04-25T13:01:18.849 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:01:18.849 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:01:18.849 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2024-04-25T13:01:18.849 INFO:teuthology.orchestra.run.smithi066.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:01:18.849 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:01:18.849 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:01:18.850 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:01:18.850 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:01:18.850 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:01:18.850 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-04-25T13:01:18.872 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-04-25T13:01:18.890 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-04-25T13:01:18.890 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-04-25T13:01:18.946 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:01:18.947 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:01:18.948 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-04-25T13:01:18.964 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T13:01:18.974 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T13:01:18.980 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T13:01:18.980 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:01:18.982 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:01:18.984 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:01:19.002 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T13:01:19.008 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T13:01:19.008 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:01:19.039 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:01:19.042 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:01:19.131 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T13:01:19.203 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T13:01:19.218 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T13:01:19.218 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T13:01:19.218 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T13:01:19.218 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:01:19.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:01:19.235 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-04-25T13:01:19.293 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-437.el9.x86_64 2024-04-25T13:01:19.293 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-437.el9.x86_64 2024-04-25T13:01:19.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-04-25T13:01:26.150 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 1.5 MB/s | 8.0 MB 00:05 2024-04-25T13:01:30.359 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 19 MB 00:03 2024-04-25T13:01:51.974 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 352 kB/s | 6.2 MB 00:17 2024-04-25T13:02:01.988 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 1.9 kB/s | 16 kB 00:08 2024-04-25T13:02:04.589 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 21 MB 00:02 2024-04-25T13:02:09.780 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-25T13:02:11.744 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T13:02:11.745 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T13:02:11.824 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-25T13:02:11.825 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-25T13:02:11.825 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-25T13:02:11.935 DEBUG:teuthology.orchestra.run.smithi031:> echo no | sudo yum reinstall kernel || true 2024-04-25T13:02:12.458 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:02:09 PM UTC. 2024-04-25T13:02:12.583 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-25T13:02:12.597 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T13:02:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-25T13:02:12.597 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T13:02:12.597 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalling: 2024-04-25T13:02:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T13:02:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:02:12.597 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-25T13:02:12.597 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T13:02:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:02:12.598 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 255 k 2024-04-25T13:02:12.598 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 0 2024-04-25T13:02:12.598 INFO:teuthology.orchestra.run.smithi031.stderr:Operation aborted. 2024-04-25T13:02:12.650 INFO:teuthology.orchestra.run.smithi031.stdout:Is this ok [y/N]: 2024-04-25T13:02:12.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum reinstall -y kernel || true 2024-04-25T13:02:13.166 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 01:02:09 PM UTC. 2024-04-25T13:02:13.286 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-25T13:02:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T13:02:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-25T13:02:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T13:02:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalling: 2024-04-25T13:02:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T13:02:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:02:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-25T13:02:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-25T13:02:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:02:13.287 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 255 k 2024-04-25T13:02:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 0 2024-04-25T13:02:13.288 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-25T13:02:21.964 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.7 MB/s | 255 kB 00:00 2024-04-25T13:02:21.964 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:02:21.965 INFO:teuthology.orchestra.run.smithi031.stdout:Total 29 kB/s | 255 kB 00:08 2024-04-25T13:02:21.985 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-25T13:02:22.205 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-25T13:02:22.205 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-25T13:02:22.210 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-25T13:02:22.211 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-25T13:02:22.339 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-25T13:02:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T13:02:22.415 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:02:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:02:22.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T13:02:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:02:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:02:23.698 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalled: 2024-04-25T13:02:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T13:02:23.698 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:02:23.698 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-25T13:02:23.878 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T13:02:23.925 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T13:02:23.926 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T13:02:23.926 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T13:02:23.927 DEBUG:teuthology.parallel:result is None 2024-04-25T13:02:23.927 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T13:02:23.934 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T13:02:23.934 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:02:23.936 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:02:23.938 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:02:23.956 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T13:02:23.962 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T13:02:23.973 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T13:02:23.973 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:02:23.991 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:02:23.996 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:02:24.030 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T13:02:24.037 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T13:02:24.037 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:02:24.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:02:24.079 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:02:24.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:02:24.095 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:02:24.098 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:02:24.122 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:02:24.149 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:02:24.159 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:02:24.166 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:02:24.180 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:02:24.180 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:02:24.189 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:02:24.191 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T13:02:24.198 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T13:02:24.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:02:24.200 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:02:24.234 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:02:24.261 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T13:02:24.270 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T13:02:24.270 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:02:24.273 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:02:24.301 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:02:24.317 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:02:24.367 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:02:24.416 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:02:24.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:02:24.537 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:02:24.537 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:02:24.601 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:02:24.645 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:02:24.697 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:02:24.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:02:24.826 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-25T13:02:24.826 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:02:24.894 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:02:24.937 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:02:24.988 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:02:25.073 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:02:25.114 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T13:02:25.114 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:02:25.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-04-25T13:02:25.183 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-04-25T13:02:25.185 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-04-25T13:02:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:02:25.226 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:02:25.263 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:02:25.755 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T13:02:25.764 INFO:teuthology.task.internal:Starting timer... 2024-04-25T13:02:25.764 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T13:02:25.774 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T13:02:25.782 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T13:02:25.782 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-04-25T13:02:25.832 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-04-25T13:02:25.835 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2024-04-25T13:02:25.914 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2024-04-25T13:02:25.916 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-04-25T13:02:25.994 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-04-25T13:02:25.996 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T13:02:25.997 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-04-25T13:02:26.019 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-04-25T13:02:26.019 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2024-04-25T13:02:26.038 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2024-04-25T13:02:26.039 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-04-25T13:02:26.060 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-04-25T13:02:26.060 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive'} 2024-04-25T13:02:26.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:02:26.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:02:26.152 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:02:26.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:02:26.181 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:02:26.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:02:26.216 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T13:02:26.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:02:26.242 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:02:26.283 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:02:26.311 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T13:02:26.395 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T13:02:26.400 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T13:02:26.613 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T13:02:26.629 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T13:02:26.630 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi066.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T13:06:30.584 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2024-04-25T13:06:30.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-25T13:06:30.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:06:30.661 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-04-25T13:06:30.742 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-25T13:06:30.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-25T13:06:30.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:06:30.816 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-04-25T13:06:30.895 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-04-25T13:06:30.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-25T13:06:30.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:06:30.970 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-04-25T13:06:31.054 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-25T13:06:31.055 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T13:06:31.067 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T13:06:31.067 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:06:31.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:06:31.070 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:06:31.070 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:06:31.072 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:06:31.073 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:06:31.102 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:06:31.105 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:06:31.117 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:06:31.120 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:06:31.126 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:06:31.140 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:06:31.142 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-04-25T13:06:31.145 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2024-04-25T13:06:31.154 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-04-25T13:06:31.157 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2024-04-25T13:06:31.163 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-04-25T13:06:31.168 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:06:31.171 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:06:31.174 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-04-25T13:06:31.182 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:06:31.185 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:06:31.189 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:06:31.201 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:06:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:06:31.242 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:06:31.244 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:06:31.244 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-04-25T13:06:31.244 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.244 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.244 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.244 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.245 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:06:31.246 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-04-25T13:06:31.246 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.246 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.246 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.246 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.260 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:06:31.263 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:06:31.263 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-04-25T13:06:31.263 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.263 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.263 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.264 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:06:31.264 INFO:teuthology.run_tasks:Running task install... 2024-04-25T13:06:31.273 DEBUG:teuthology.task.install:project ceph 2024-04-25T13:06:31.273 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T13:06:31.273 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T13:06:31.273 INFO:teuthology.task.install:Using flavor: default 2024-04-25T13:06:31.279 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T13:06:31.279 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T13:06:31.279 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T13:06:31.280 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:06:31.282 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T13:06:31.282 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:06:31.283 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T13:06:31.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:06:31.422 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T13:06:31.423 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T13:06:31.430 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T13:06:31.431 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T13:06:31.510 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T13:06:31.510 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T13:06:31.537 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:06:31.538 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:06:31.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:06:31.565 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:06:31.565 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:06:31.607 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:06:31.608 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-25T13:06:31.608 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:06:31.634 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:06:31.634 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:06:31.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:06:31.683 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:06:31.683 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T13:06:31.683 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:06:31.700 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:06:31.707 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:06:31.708 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:06:31.711 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:06:31.772 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:06:31.774 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:06:31.776 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-04-25T13:06:31.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-04-25T13:06:31.837 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2024-04-25T13:06:31.839 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2024-04-25T13:06:31.844 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:06:31.907 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2024-04-25T13:06:31.909 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2024-04-25T13:06:32.012 INFO:teuthology.orchestra.run.smithi031.stdout:48 files removed 2024-04-25T13:06:32.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:06:32.072 INFO:teuthology.orchestra.run.smithi066.stdout:48 files removed 2024-04-25T13:06:32.099 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:06:32.132 INFO:teuthology.orchestra.run.smithi136.stdout:48 files removed 2024-04-25T13:06:32.159 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:06:32.659 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 239 kB/s | 83 kB 00:00 2024-04-25T13:06:32.690 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 262 kB/s | 83 kB 00:00 2024-04-25T13:06:32.911 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 170 kB/s | 83 kB 00:00 2024-04-25T13:06:33.043 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-04-25T13:06:33.115 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-25T13:06:33.229 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-04-25T13:06:33.482 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-25T13:06:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-25T13:06:33.696 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-25T13:06:34.062 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T13:06:34.376 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.0 MB 00:00 2024-04-25T13:06:36.080 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T13:06:40.281 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T13:06:42.171 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-25T13:06:43.905 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T13:06:44.113 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 794 kB/s | 8.0 MB 00:10 2024-04-25T13:06:45.915 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-25T13:06:46.177 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 1.8 MB/s | 19 MB 00:10 2024-04-25T13:06:49.180 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-04-25T13:06:50.132 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-04-25T13:06:50.494 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T13:06:51.374 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:06:51.374 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:06:51.385 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:06:51.386 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:06:51.387 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:06:51.388 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:06:51.390 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:06:51.392 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:06:51.392 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:06:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-25T13:06:51.466 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-04-25T13:06:51.466 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-25T13:06:51.466 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-04-25T13:06:51.466 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-25T13:06:51.467 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:06:51.467 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T13:06:51.467 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:06:51.467 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T13:06:51.467 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T13:06:51.467 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T13:06:51.467 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T13:06:51.467 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T13:06:51.467 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T13:06:51.467 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T13:06:51.467 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T13:06:51.468 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T13:06:51.469 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T13:06:51.470 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T13:06:51.471 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T13:06:51.471 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T13:06:51.471 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T13:06:51.471 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T13:06:51.471 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T13:06:51.471 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:06:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:06:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:06:51.474 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:06:51.475 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:06:51.476 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:06:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:06:51.478 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:06:51.479 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:06:51.479 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:06:51.479 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:06:51.479 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-04-25T13:06:51.479 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:06:51.479 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:06:51.479 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-25T13:06:51.479 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-04-25T13:06:51.479 INFO:teuthology.orchestra.run.smithi031.stdout:Install 148 Packages 2024-04-25T13:06:51.479 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 4 Packages 2024-04-25T13:06:51.479 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:06:51.480 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 222 M 2024-04-25T13:06:51.480 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-25T13:06:51.588 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-25T13:06:52.101 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-04-25T13:06:53.293 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T13:06:53.781 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T13:06:58.448 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-25T13:06:59.094 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-25T13:07:00.566 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:07:00.567 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:07:00.577 INFO:teuthology.orchestra.run.smithi136.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:07:00.578 INFO:teuthology.orchestra.run.smithi136.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:07:00.579 INFO:teuthology.orchestra.run.smithi136.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:07:00.580 INFO:teuthology.orchestra.run.smithi136.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:07:00.581 INFO:teuthology.orchestra.run.smithi136.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:07:00.583 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:07:00.584 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:07:00.645 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-25T13:07:00.661 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-04-25T13:07:00.661 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-25T13:07:00.661 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-04-25T13:07:00.662 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-25T13:07:00.662 INFO:teuthology.orchestra.run.smithi136.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:07:00.662 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T13:07:00.662 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:07:00.662 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T13:07:00.662 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T13:07:00.662 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T13:07:00.662 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T13:07:00.662 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T13:07:00.662 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T13:07:00.662 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T13:07:00.663 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T13:07:00.663 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T13:07:00.663 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T13:07:00.663 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T13:07:00.663 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T13:07:00.663 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T13:07:00.663 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T13:07:00.663 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:07:00.663 INFO:teuthology.orchestra.run.smithi136.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:07:00.664 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:07:00.664 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T13:07:00.664 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T13:07:00.664 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T13:07:00.664 INFO:teuthology.orchestra.run.smithi136.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:07:00.664 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:07:00.664 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:07:00.664 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:07:00.664 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:07:00.664 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:07:00.665 INFO:teuthology.orchestra.run.smithi136.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:07:00.666 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:07:00.666 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-25T13:07:00.666 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:07:00.666 INFO:teuthology.orchestra.run.smithi136.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:07:00.666 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:07:00.666 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T13:07:00.666 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T13:07:00.666 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T13:07:00.667 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T13:07:00.667 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T13:07:00.667 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T13:07:00.667 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T13:07:00.667 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T13:07:00.667 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:07:00.667 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:07:00.667 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:07:00.667 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:07:00.668 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:07:00.668 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:07:00.668 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:07:00.668 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:07:00.668 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T13:07:00.668 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:07:00.668 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:07:00.668 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:07:00.669 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:07:00.669 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:07:00.669 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T13:07:00.669 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:07:00.669 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:07:00.669 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:07:00.669 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:07:00.669 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:07:00.669 INFO:teuthology.orchestra.run.smithi136.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:07:00.669 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:07:00.669 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T13:07:00.670 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:07:00.671 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:07:00.672 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:07:00.673 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:07:00.674 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout:Install 148 Packages 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 4 Packages 2024-04-25T13:07:00.675 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:00.676 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 222 M 2024-04-25T13:07:00.676 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-25T13:07:01.166 INFO:teuthology.orchestra.run.smithi136.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-25T13:07:01.321 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:07:01.346 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:07:01.346 INFO:teuthology.orchestra.run.smithi066.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:07:01.346 INFO:teuthology.orchestra.run.smithi066.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:07:01.346 INFO:teuthology.orchestra.run.smithi066.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:07:01.346 INFO:teuthology.orchestra.run.smithi066.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:07:01.346 INFO:teuthology.orchestra.run.smithi066.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:07:01.346 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:07:01.346 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:07:01.405 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-04-25T13:07:01.483 INFO:teuthology.orchestra.run.smithi136.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.7 MB/s | 890 kB 00:00 2024-04-25T13:07:01.505 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-25T13:07:01.505 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-04-25T13:07:01.505 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T13:07:01.506 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T13:07:01.507 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T13:07:01.507 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T13:07:01.507 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T13:07:01.507 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T13:07:01.507 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:07:01.507 INFO:teuthology.orchestra.run.smithi066.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:07:01.507 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:07:01.507 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T13:07:01.507 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T13:07:01.508 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:07:01.509 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:07:01.510 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:07:01.511 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:07:01.512 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:07:01.512 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:07:01.512 INFO:teuthology.orchestra.run.smithi066.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:07:01.512 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:07:01.512 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:07:01.512 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:07:01.512 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:07:01.512 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:07:01.512 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:07:01.512 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:07:01.512 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:07:01.513 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:07:01.513 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:07:01.513 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:07:01.513 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:07:01.513 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:07:01.513 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T13:07:01.513 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T13:07:01.513 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:07:01.514 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:07:01.514 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:07:01.514 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:07:01.514 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:07:01.514 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:07:01.514 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:07:01.514 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:07:01.514 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:07:01.514 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:07:01.514 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:07:01.514 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:07:01.515 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:07:01.515 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:07:01.515 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:07:01.515 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:07:01.515 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:07:01.515 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:07:01.515 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:07:01.515 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:07:01.515 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:07:01.515 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:07:01.516 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:07:01.516 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:07:01.516 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:07:01.516 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:07:01.516 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:07:01.516 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:07:01.516 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:07:01.516 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:07:01.517 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:07:01.518 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:07:01.518 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:07:01.518 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:07:01.518 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:07:01.518 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:07:01.518 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:07:01.518 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:07:01.518 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:07:01.518 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout:Installing weak dependencies: 2024-04-25T13:07:01.519 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:07:01.520 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:07:01.520 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-04-25T13:07:01.520 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-04-25T13:07:01.520 INFO:teuthology.orchestra.run.smithi066.stdout:Install 148 Packages 2024-04-25T13:07:01.520 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 4 Packages 2024-04-25T13:07:01.520 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:07:01.520 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 222 M 2024-04-25T13:07:01.520 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-04-25T13:07:01.567 INFO:teuthology.orchestra.run.smithi136.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.7 MB/s | 145 kB 00:00 2024-04-25T13:07:01.922 INFO:teuthology.orchestra.run.smithi031.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-25T13:07:02.042 INFO:teuthology.orchestra.run.smithi136.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 4.9 MB/s | 5.4 MB 00:01 2024-04-25T13:07:02.100 INFO:teuthology.orchestra.run.smithi136.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 4.4 MB/s | 2.4 MB 00:00 2024-04-25T13:07:02.417 INFO:teuthology.orchestra.run.smithi136.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 4.0 MB/s | 1.5 MB 00:00 2024-04-25T13:07:02.697 INFO:teuthology.orchestra.run.smithi031.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 890 kB 00:00 2024-04-25T13:07:02.759 INFO:teuthology.orchestra.run.smithi136.stdout:(7/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 12 MB/s | 21 MB 00:01 2024-04-25T13:07:02.789 INFO:teuthology.orchestra.run.smithi031.stdout:(3/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 4.4 MB/s | 5.4 MB 00:01 2024-04-25T13:07:02.864 INFO:teuthology.orchestra.run.smithi031.stdout:(4/152): ceph-immutable-object-cache-19.0.0-315 869 kB/s | 145 kB 00:00 2024-04-25T13:07:03.189 INFO:teuthology.orchestra.run.smithi031.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 5.9 MB/s | 2.4 MB 00:00 2024-04-25T13:07:03.656 INFO:teuthology.orchestra.run.smithi031.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 1.9 MB/s | 1.5 MB 00:00 2024-04-25T13:07:03.726 INFO:teuthology.orchestra.run.smithi136.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 11 MB/s | 11 MB 00:00 2024-04-25T13:07:03.793 INFO:teuthology.orchestra.run.smithi136.stdout:(9/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.8 MB/s | 4.7 MB 00:01 2024-04-25T13:07:03.818 INFO:teuthology.orchestra.run.smithi136.stdout:(10/152): ceph-selinux-19.0.0-3156.gd69f13d8.el 274 kB/s | 25 kB 00:00 2024-04-25T13:07:03.893 INFO:teuthology.orchestra.run.smithi136.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.9 MB/s | 225 kB 00:00 2024-04-25T13:07:03.960 INFO:teuthology.orchestra.run.smithi136.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 405 kB/s | 27 kB 00:00 2024-04-25T13:07:04.019 INFO:teuthology.orchestra.run.smithi136.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 543 kB/s | 32 kB 00:00 2024-04-25T13:07:04.040 INFO:teuthology.orchestra.run.smithi031.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.5 MB/s | 4.7 MB 00:00 2024-04-25T13:07:04.111 INFO:teuthology.orchestra.run.smithi136.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 7.9 MB/s | 747 kB 00:00 2024-04-25T13:07:04.186 INFO:teuthology.orchestra.run.smithi136.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.1 MB/s | 163 kB 00:00 2024-04-25T13:07:04.262 INFO:teuthology.orchestra.run.smithi136.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.6 MB/s | 126 kB 00:00 2024-04-25T13:07:04.354 INFO:teuthology.orchestra.run.smithi136.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 5.3 MB/s | 500 kB 00:00 2024-04-25T13:07:05.013 INFO:teuthology.orchestra.run.smithi136.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 8.1 MB/s | 5.4 MB 00:00 2024-04-25T13:07:05.104 INFO:teuthology.orchestra.run.smithi136.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 493 kB/s | 45 kB 00:00 2024-04-25T13:07:05.180 INFO:teuthology.orchestra.run.smithi136.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.8 MB/s | 135 kB 00:00 2024-04-25T13:07:05.255 INFO:teuthology.orchestra.run.smithi136.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.1 MB/s | 161 kB 00:00 2024-04-25T13:07:05.339 INFO:teuthology.orchestra.run.smithi136.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.8 MB/s | 320 kB 00:00 2024-04-25T13:07:05.422 INFO:teuthology.orchestra.run.smithi136.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 298 kB 00:00 2024-04-25T13:07:05.498 INFO:teuthology.orchestra.run.smithi136.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 99 kB 00:00 2024-04-25T13:07:05.581 INFO:teuthology.orchestra.run.smithi136.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 85 kB 00:00 2024-04-25T13:07:05.873 INFO:teuthology.orchestra.run.smithi136.stdout:(26/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 4.9 MB/s | 17 MB 00:03 2024-04-25T13:07:05.957 INFO:teuthology.orchestra.run.smithi136.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 170 kB 00:00 2024-04-25T13:07:06.007 INFO:teuthology.orchestra.run.smithi136.stdout:(28/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 7.3 MB/s | 3.1 MB 00:00 2024-04-25T13:07:06.024 INFO:teuthology.orchestra.run.smithi136.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 511 kB/s | 34 kB 00:00 2024-04-25T13:07:06.058 INFO:teuthology.orchestra.run.smithi136.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.8 MB/s | 142 kB 00:00 2024-04-25T13:07:06.073 INFO:teuthology.orchestra.run.smithi031.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 7.0 MB/s | 17 MB 00:02 2024-04-25T13:07:06.190 INFO:teuthology.orchestra.run.smithi031.stdout:(9/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 4.9 MB/s | 11 MB 00:02 2024-04-25T13:07:06.215 INFO:teuthology.orchestra.run.smithi031.stdout:(10/152): ceph-selinux-19.0.0-3156.gd69f13d8.el 177 kB/s | 25 kB 00:00 2024-04-25T13:07:06.316 INFO:teuthology.orchestra.run.smithi031.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.2 MB/s | 225 kB 00:00 2024-04-25T13:07:06.399 INFO:teuthology.orchestra.run.smithi031.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 324 kB/s | 27 kB 00:00 2024-04-25T13:07:06.492 INFO:teuthology.orchestra.run.smithi031.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 345 kB/s | 32 kB 00:00 2024-04-25T13:07:06.659 INFO:teuthology.orchestra.run.smithi136.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 5.6 MB/s | 3.5 MB 00:00 2024-04-25T13:07:06.659 INFO:teuthology.orchestra.run.smithi031.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 4.4 MB/s | 747 kB 00:00 2024-04-25T13:07:06.751 INFO:teuthology.orchestra.run.smithi031.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 163 kB 00:00 2024-04-25T13:07:06.759 INFO:teuthology.orchestra.run.smithi136.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.4 MB/s | 249 kB 00:00 2024-04-25T13:07:06.817 INFO:teuthology.orchestra.run.smithi031.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.9 MB/s | 126 kB 00:00 2024-04-25T13:07:06.842 INFO:teuthology.orchestra.run.smithi136.stdout:(33/152): ceph-mgr-diskprediction-local-19.0.0- 9.4 MB/s | 7.4 MB 00:00 2024-04-25T13:07:06.868 INFO:teuthology.orchestra.run.smithi136.stdout:(34/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 454 kB/s | 49 kB 00:00 2024-04-25T13:07:06.901 INFO:teuthology.orchestra.run.smithi031.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 5.9 MB/s | 500 kB 00:00 2024-04-25T13:07:06.902 INFO:teuthology.orchestra.run.smithi136.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 280 kB/s | 16 kB 00:00 2024-04-25T13:07:06.936 INFO:teuthology.orchestra.run.smithi136.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 4.0 MB/s | 273 kB 00:00 2024-04-25T13:07:07.035 INFO:teuthology.orchestra.run.smithi136.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 5.6 MB/s | 762 kB 00:00 2024-04-25T13:07:07.060 INFO:teuthology.orchestra.run.smithi136.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 146 kB/s | 18 kB 00:00 2024-04-25T13:07:07.185 INFO:teuthology.orchestra.run.smithi136.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-04-25T13:07:07.245 INFO:teuthology.orchestra.run.smithi136.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-25T13:07:07.269 INFO:teuthology.orchestra.run.smithi136.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.7 MB/s | 803 kB 00:00 2024-04-25T13:07:07.294 INFO:teuthology.orchestra.run.smithi136.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T13:07:07.337 INFO:teuthology.orchestra.run.smithi136.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T13:07:07.362 INFO:teuthology.orchestra.run.smithi136.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T13:07:07.388 INFO:teuthology.orchestra.run.smithi136.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T13:07:07.409 INFO:teuthology.orchestra.run.smithi031.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 11 MB/s | 5.4 MB 00:00 2024-04-25T13:07:07.412 INFO:teuthology.orchestra.run.smithi136.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T13:07:07.446 INFO:teuthology.orchestra.run.smithi136.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T13:07:07.468 INFO:teuthology.orchestra.run.smithi031.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 775 kB/s | 45 kB 00:00 2024-04-25T13:07:07.480 INFO:teuthology.orchestra.run.smithi136.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-04-25T13:07:07.513 INFO:teuthology.orchestra.run.smithi136.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T13:07:07.535 INFO:teuthology.orchestra.run.smithi031.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.0 MB/s | 135 kB 00:00 2024-04-25T13:07:07.539 INFO:teuthology.orchestra.run.smithi136.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T13:07:07.556 INFO:teuthology.orchestra.run.smithi136.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 514 kB/s | 21 kB 00:00 2024-04-25T13:07:07.581 INFO:teuthology.orchestra.run.smithi136.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-04-25T13:07:07.598 INFO:teuthology.orchestra.run.smithi136.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 788 kB/s | 33 kB 00:00 2024-04-25T13:07:07.602 INFO:teuthology.orchestra.run.smithi031.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.4 MB/s | 161 kB 00:00 2024-04-25T13:07:07.632 INFO:teuthology.orchestra.run.smithi136.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 574 kB/s | 18 kB 00:00 2024-04-25T13:07:07.686 INFO:teuthology.orchestra.run.smithi031.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.8 MB/s | 320 kB 00:00 2024-04-25T13:07:07.690 INFO:teuthology.orchestra.run.smithi136.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 27 MB/s | 3.0 MB 00:00 2024-04-25T13:07:07.707 INFO:teuthology.orchestra.run.smithi136.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 151 kB/s | 11 kB 00:00 2024-04-25T13:07:07.732 INFO:teuthology.orchestra.run.smithi136.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-04-25T13:07:07.749 INFO:teuthology.orchestra.run.smithi136.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T13:07:07.761 INFO:teuthology.orchestra.run.smithi031.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.9 MB/s | 298 kB 00:00 2024-04-25T13:07:07.783 INFO:teuthology.orchestra.run.smithi136.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T13:07:07.808 INFO:teuthology.orchestra.run.smithi136.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T13:07:07.819 INFO:teuthology.orchestra.run.smithi031.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 99 kB 00:00 2024-04-25T13:07:07.825 INFO:teuthology.orchestra.run.smithi136.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 618 kB/s | 25 kB 00:00 2024-04-25T13:07:07.842 INFO:teuthology.orchestra.run.smithi136.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T13:07:07.876 INFO:teuthology.orchestra.run.smithi136.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T13:07:07.878 INFO:teuthology.orchestra.run.smithi031.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-25T13:07:07.901 INFO:teuthology.orchestra.run.smithi136.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T13:07:07.927 INFO:teuthology.orchestra.run.smithi136.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-04-25T13:07:08.061 INFO:teuthology.orchestra.run.smithi136.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-25T13:07:08.136 INFO:teuthology.orchestra.run.smithi136.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-25T13:07:08.178 INFO:teuthology.orchestra.run.smithi136.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-25T13:07:08.203 INFO:teuthology.orchestra.run.smithi031.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 9.5 MB/s | 3.1 MB 00:00 2024-04-25T13:07:08.220 INFO:teuthology.orchestra.run.smithi136.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T13:07:08.245 INFO:teuthology.orchestra.run.smithi136.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T13:07:08.287 INFO:teuthology.orchestra.run.smithi031.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 170 kB 00:00 2024-04-25T13:07:08.354 INFO:teuthology.orchestra.run.smithi031.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 515 kB/s | 34 kB 00:00 2024-04-25T13:07:08.370 INFO:teuthology.orchestra.run.smithi136.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 39 MB/s | 5.8 MB 00:00 2024-04-25T13:07:08.404 INFO:teuthology.orchestra.run.smithi136.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.5 MB/s | 1.0 MB 00:00 2024-04-25T13:07:08.429 INFO:teuthology.orchestra.run.smithi031.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.9 MB/s | 142 kB 00:00 2024-04-25T13:07:08.429 INFO:teuthology.orchestra.run.smithi136.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T13:07:08.566 INFO:teuthology.orchestra.run.smithi136.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-04-25T13:07:08.588 INFO:teuthology.orchestra.run.smithi136.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-25T13:07:08.614 INFO:teuthology.orchestra.run.smithi136.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T13:07:08.639 INFO:teuthology.orchestra.run.smithi136.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-04-25T13:07:08.656 INFO:teuthology.orchestra.run.smithi136.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 458 kB/s | 19 kB 00:00 2024-04-25T13:07:08.681 INFO:teuthology.orchestra.run.smithi136.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T13:07:08.706 INFO:teuthology.orchestra.run.smithi136.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-04-25T13:07:08.765 INFO:teuthology.orchestra.run.smithi136.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-25T13:07:08.804 INFO:teuthology.orchestra.run.smithi031.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 9.4 MB/s | 3.5 MB 00:00 2024-04-25T13:07:08.840 INFO:teuthology.orchestra.run.smithi136.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-25T13:07:08.865 INFO:teuthology.orchestra.run.smithi136.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-25T13:07:08.891 INFO:teuthology.orchestra.run.smithi136.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T13:07:08.908 INFO:teuthology.orchestra.run.smithi136.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-25T13:07:08.933 INFO:teuthology.orchestra.run.smithi136.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-04-25T13:07:08.958 INFO:teuthology.orchestra.run.smithi136.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T13:07:08.984 INFO:teuthology.orchestra.run.smithi136.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T13:07:09.009 INFO:teuthology.orchestra.run.smithi136.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T13:07:09.034 INFO:teuthology.orchestra.run.smithi136.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T13:07:09.145 INFO:teuthology.orchestra.run.smithi066.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-25T13:07:09.235 INFO:teuthology.orchestra.run.smithi136.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 211 kB/s | 42 kB 00:00 2024-04-25T13:07:09.527 INFO:teuthology.orchestra.run.smithi136.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-04-25T13:07:09.538 INFO:teuthology.orchestra.run.smithi031.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-25T13:07:09.552 INFO:teuthology.orchestra.run.smithi136.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 140 kB/s | 44 kB 00:00 2024-04-25T13:07:09.586 INFO:teuthology.orchestra.run.smithi136.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-04-25T13:07:09.611 INFO:teuthology.orchestra.run.smithi136.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T13:07:09.636 INFO:teuthology.orchestra.run.smithi136.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-25T13:07:09.670 INFO:teuthology.orchestra.run.smithi136.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T13:07:09.695 INFO:teuthology.orchestra.run.smithi136.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 996 kB/s | 58 kB 00:00 2024-04-25T13:07:09.704 INFO:teuthology.orchestra.run.smithi066.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 890 kB 00:00 2024-04-25T13:07:09.720 INFO:teuthology.orchestra.run.smithi136.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T13:07:09.746 INFO:teuthology.orchestra.run.smithi136.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T13:07:09.771 INFO:teuthology.orchestra.run.smithi136.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-04-25T13:07:09.779 INFO:teuthology.orchestra.run.smithi066.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.9 MB/s | 145 kB 00:00 2024-04-25T13:07:09.796 INFO:teuthology.orchestra.run.smithi136.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-25T13:07:09.838 INFO:teuthology.orchestra.run.smithi136.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-25T13:07:09.847 INFO:teuthology.orchestra.run.smithi031.stdout:(32/152): ceph-common-19.0.0-3156.gd69f13d8.el7 2.6 MB/s | 21 MB 00:08 2024-04-25T13:07:09.972 INFO:teuthology.orchestra.run.smithi031.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3156.gd6 574 kB/s | 249 kB 00:00 2024-04-25T13:07:09.989 INFO:teuthology.orchestra.run.smithi031.stdout:(34/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 348 kB/s | 49 kB 00:00 2024-04-25T13:07:10.047 INFO:teuthology.orchestra.run.smithi031.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 219 kB/s | 16 kB 00:00 2024-04-25T13:07:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.8 MB/s | 273 kB 00:00 2024-04-25T13:07:10.240 INFO:teuthology.orchestra.run.smithi031.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 3.9 MB/s | 762 kB 00:00 2024-04-25T13:07:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 156 kB/s | 18 kB 00:00 2024-04-25T13:07:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-04-25T13:07:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-04-25T13:07:10.455 INFO:teuthology.orchestra.run.smithi136.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 501 kB/s | 309 kB 00:00 2024-04-25T13:07:10.457 INFO:teuthology.orchestra.run.smithi031.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-25T13:07:10.482 INFO:teuthology.orchestra.run.smithi031.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T13:07:10.521 INFO:teuthology.orchestra.run.smithi066.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.2 MB/s | 2.4 MB 00:00 2024-04-25T13:07:10.524 INFO:teuthology.orchestra.run.smithi031.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T13:07:10.541 INFO:teuthology.orchestra.run.smithi031.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T13:07:10.555 INFO:teuthology.orchestra.run.smithi136.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 724 kB/s | 548 kB 00:00 2024-04-25T13:07:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T13:07:10.580 INFO:teuthology.orchestra.run.smithi136.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-04-25T13:07:10.592 INFO:teuthology.orchestra.run.smithi031.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T13:07:10.617 INFO:teuthology.orchestra.run.smithi031.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T13:07:10.651 INFO:teuthology.orchestra.run.smithi031.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-04-25T13:07:10.681 INFO:teuthology.orchestra.run.smithi136.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-25T13:07:10.684 INFO:teuthology.orchestra.run.smithi031.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T13:07:10.710 INFO:teuthology.orchestra.run.smithi031.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T13:07:10.735 INFO:teuthology.orchestra.run.smithi031.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-25T13:07:10.765 INFO:teuthology.orchestra.run.smithi031.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T13:07:10.781 INFO:teuthology.orchestra.run.smithi136.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-04-25T13:07:10.794 INFO:teuthology.orchestra.run.smithi031.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 561 kB/s | 33 kB 00:00 2024-04-25T13:07:10.796 INFO:teuthology.orchestra.run.smithi066.stdout:(5/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.7 MB/s | 5.4 MB 00:01 2024-04-25T13:07:10.869 INFO:teuthology.orchestra.run.smithi031.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-25T13:07:10.890 INFO:teuthology.orchestra.run.smithi136.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-04-25T13:07:10.894 INFO:teuthology.orchestra.run.smithi031.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 187 kB/s | 18 kB 00:00 2024-04-25T13:07:10.920 INFO:teuthology.orchestra.run.smithi031.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-25T13:07:10.945 INFO:teuthology.orchestra.run.smithi031.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T13:07:10.946 INFO:teuthology.orchestra.run.smithi066.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.5 MB/s | 1.5 MB 00:00 2024-04-25T13:07:10.965 INFO:teuthology.orchestra.run.smithi136.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T13:07:10.970 INFO:teuthology.orchestra.run.smithi031.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-04-25T13:07:10.995 INFO:teuthology.orchestra.run.smithi031.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T13:07:11.015 INFO:teuthology.orchestra.run.smithi136.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-25T13:07:11.029 INFO:teuthology.orchestra.run.smithi031.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T13:07:11.054 INFO:teuthology.orchestra.run.smithi031.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 441 kB/s | 25 kB 00:00 2024-04-25T13:07:11.079 INFO:teuthology.orchestra.run.smithi031.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-04-25T13:07:11.091 INFO:teuthology.orchestra.run.smithi136.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-04-25T13:07:11.105 INFO:teuthology.orchestra.run.smithi031.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T13:07:11.130 INFO:teuthology.orchestra.run.smithi031.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T13:07:11.155 INFO:teuthology.orchestra.run.smithi031.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-04-25T13:07:11.182 INFO:teuthology.orchestra.run.smithi136.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 476 kB/s | 43 kB 00:00 2024-04-25T13:07:11.241 INFO:teuthology.orchestra.run.smithi136.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-04-25T13:07:11.256 INFO:teuthology.orchestra.run.smithi031.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-25T13:07:11.275 INFO:teuthology.orchestra.run.smithi136.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-04-25T13:07:11.334 INFO:teuthology.orchestra.run.smithi136.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-25T13:07:11.339 INFO:teuthology.orchestra.run.smithi031.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-04-25T13:07:11.373 INFO:teuthology.orchestra.run.smithi031.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-25T13:07:11.375 INFO:teuthology.orchestra.run.smithi136.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T13:07:11.415 INFO:teuthology.orchestra.run.smithi031.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T13:07:11.440 INFO:teuthology.orchestra.run.smithi031.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T13:07:11.467 INFO:teuthology.orchestra.run.smithi136.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-25T13:07:11.493 INFO:teuthology.orchestra.run.smithi136.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-25T13:07:11.522 INFO:teuthology.orchestra.run.smithi066.stdout:(7/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 7.8 MB/s | 21 MB 00:02 2024-04-25T13:07:11.559 INFO:teuthology.orchestra.run.smithi136.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-04-25T13:07:11.565 INFO:teuthology.orchestra.run.smithi031.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 39 MB/s | 5.8 MB 00:00 2024-04-25T13:07:11.599 INFO:teuthology.orchestra.run.smithi031.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.5 MB/s | 1.0 MB 00:00 2024-04-25T13:07:11.609 INFO:teuthology.orchestra.run.smithi136.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T13:07:11.624 INFO:teuthology.orchestra.run.smithi031.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T13:07:11.710 INFO:teuthology.orchestra.run.smithi136.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-25T13:07:11.722 INFO:teuthology.orchestra.run.smithi066.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.1 MB/s | 4.7 MB 00:00 2024-04-25T13:07:11.749 INFO:teuthology.orchestra.run.smithi031.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-04-25T13:07:11.791 INFO:teuthology.orchestra.run.smithi031.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-25T13:07:11.810 INFO:teuthology.orchestra.run.smithi136.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-04-25T13:07:11.816 INFO:teuthology.orchestra.run.smithi031.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-04-25T13:07:11.822 INFO:teuthology.orchestra.run.smithi066.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 252 kB/s | 25 kB 00:00 2024-04-25T13:07:11.838 INFO:teuthology.orchestra.run.smithi136.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-04-25T13:07:11.842 INFO:teuthology.orchestra.run.smithi031.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-04-25T13:07:11.867 INFO:teuthology.orchestra.run.smithi031.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-25T13:07:11.892 INFO:teuthology.orchestra.run.smithi031.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T13:07:11.911 INFO:teuthology.orchestra.run.smithi136.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-25T13:07:11.917 INFO:teuthology.orchestra.run.smithi031.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 700 kB/s | 35 kB 00:00 2024-04-25T13:07:11.936 INFO:teuthology.orchestra.run.smithi136.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-25T13:07:12.012 INFO:teuthology.orchestra.run.smithi136.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-25T13:07:12.034 INFO:teuthology.orchestra.run.smithi031.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-04-25T13:07:12.037 INFO:teuthology.orchestra.run.smithi136.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-25T13:07:12.060 INFO:teuthology.orchestra.run.smithi031.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-04-25T13:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T13:07:12.110 INFO:teuthology.orchestra.run.smithi031.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T13:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-04-25T13:07:12.137 INFO:teuthology.orchestra.run.smithi136.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-25T13:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T13:07:12.171 INFO:teuthology.orchestra.run.smithi136.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-04-25T13:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-04-25T13:07:12.238 INFO:teuthology.orchestra.run.smithi136.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 787 kB/s | 79 kB 00:00 2024-04-25T13:07:12.271 INFO:teuthology.orchestra.run.smithi136.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-04-25T13:07:12.328 INFO:teuthology.orchestra.run.smithi031.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-04-25T13:07:12.347 INFO:teuthology.orchestra.run.smithi136.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-25T13:07:12.354 INFO:teuthology.orchestra.run.smithi031.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 291 kB/s | 51 kB 00:00 2024-04-25T13:07:12.372 INFO:teuthology.orchestra.run.smithi136.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-25T13:07:12.379 INFO:teuthology.orchestra.run.smithi031.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T13:07:12.437 INFO:teuthology.orchestra.run.smithi031.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 745 kB/s | 42 kB 00:00 2024-04-25T13:07:12.450 INFO:teuthology.orchestra.run.smithi136.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 900 kB/s | 90 kB 00:00 2024-04-25T13:07:12.472 INFO:teuthology.orchestra.run.smithi136.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-04-25T13:07:12.513 INFO:teuthology.orchestra.run.smithi031.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 604 kB/s | 44 kB 00:00 2024-04-25T13:07:12.564 INFO:teuthology.orchestra.run.smithi066.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 10 MB/s | 11 MB 00:01 2024-04-25T13:07:12.564 INFO:teuthology.orchestra.run.smithi136.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-04-25T13:07:12.590 INFO:teuthology.orchestra.run.smithi136.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-25T13:07:12.656 INFO:teuthology.orchestra.run.smithi066.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.4 MB/s | 225 kB 00:00 2024-04-25T13:07:12.673 INFO:teuthology.orchestra.run.smithi136.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-04-25T13:07:12.699 INFO:teuthology.orchestra.run.smithi136.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-04-25T13:07:12.714 INFO:teuthology.orchestra.run.smithi066.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 465 kB/s | 27 kB 00:00 2024-04-25T13:07:12.721 INFO:teuthology.orchestra.run.smithi031.stdout:(93/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-04-25T13:07:12.747 INFO:teuthology.orchestra.run.smithi031.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 952 kB/s | 221 kB 00:00 2024-04-25T13:07:12.772 INFO:teuthology.orchestra.run.smithi031.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 724 kB/s | 36 kB 00:00 2024-04-25T13:07:12.781 INFO:teuthology.orchestra.run.smithi066.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 476 kB/s | 32 kB 00:00 2024-04-25T13:07:12.790 INFO:teuthology.orchestra.run.smithi136.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 436 kB/s | 51 kB 00:00 2024-04-25T13:07:12.797 INFO:teuthology.orchestra.run.smithi031.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-04-25T13:07:12.816 INFO:teuthology.orchestra.run.smithi136.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-25T13:07:12.831 INFO:teuthology.orchestra.run.smithi031.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T13:07:12.858 INFO:teuthology.orchestra.run.smithi031.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 994 kB/s | 58 kB 00:00 2024-04-25T13:07:12.881 INFO:teuthology.orchestra.run.smithi031.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T13:07:12.890 INFO:teuthology.orchestra.run.smithi066.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 6.7 MB/s | 747 kB 00:00 2024-04-25T13:07:12.891 INFO:teuthology.orchestra.run.smithi136.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 896 kB/s | 90 kB 00:00 2024-04-25T13:07:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T13:07:12.925 INFO:teuthology.orchestra.run.smithi136.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-25T13:07:12.931 INFO:teuthology.orchestra.run.smithi031.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-04-25T13:07:12.957 INFO:teuthology.orchestra.run.smithi031.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-25T13:07:12.991 INFO:teuthology.orchestra.run.smithi031.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T13:07:12.991 INFO:teuthology.orchestra.run.smithi066.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.6 MB/s | 163 kB 00:00 2024-04-25T13:07:12.991 INFO:teuthology.orchestra.run.smithi136.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-25T13:07:13.025 INFO:teuthology.orchestra.run.smithi136.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T13:07:13.075 INFO:teuthology.orchestra.run.smithi066.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-25T13:07:13.134 INFO:teuthology.orchestra.run.smithi136.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T13:07:13.166 INFO:teuthology.orchestra.run.smithi066.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 5.3 MB/s | 500 kB 00:00 2024-04-25T13:07:13.408 INFO:teuthology.orchestra.run.smithi031.stdout:(104/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 6.8 MB/s | 49 MB 00:07 2024-04-25T13:07:13.584 INFO:teuthology.orchestra.run.smithi066.stdout:(18/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 6.4 MB/s | 17 MB 00:02 2024-04-25T13:07:13.649 INFO:teuthology.orchestra.run.smithi031.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-04-25T13:07:13.700 INFO:teuthology.orchestra.run.smithi066.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 387 kB/s | 45 kB 00:00 2024-04-25T13:07:13.842 INFO:teuthology.orchestra.run.smithi066.stdout:(20/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 7.9 MB/s | 5.4 MB 00:00 2024-04-25T13:07:13.868 INFO:teuthology.orchestra.run.smithi066.stdout:(21/152): python3-ceph-common-19.0.0-3156.gd69f 807 kB/s | 135 kB 00:00 2024-04-25T13:07:13.876 INFO:teuthology.orchestra.run.smithi136.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 4.2 MB/s | 3.1 MB 00:00 2024-04-25T13:07:13.943 INFO:teuthology.orchestra.run.smithi066.stdout:(22/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.6 MB/s | 161 kB 00:00 2024-04-25T13:07:13.968 INFO:teuthology.orchestra.run.smithi066.stdout:(23/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.1 MB/s | 320 kB 00:00 2024-04-25T13:07:13.999 INFO:teuthology.orchestra.run.smithi031.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 527 kB/s | 548 kB 00:01 2024-04-25T13:07:14.010 INFO:teuthology.orchestra.run.smithi066.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 4.4 MB/s | 298 kB 00:00 2024-04-25T13:07:14.025 INFO:teuthology.orchestra.run.smithi031.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 299 kB/s | 309 kB 00:01 2024-04-25T13:07:14.052 INFO:teuthology.orchestra.run.smithi066.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 99 kB 00:00 2024-04-25T13:07:14.078 INFO:teuthology.orchestra.run.smithi136.stdout:(149/152): librados2-19.0.0-3156.gd69f13d8.el7. 3.2 MB/s | 3.4 MB 00:01 2024-04-25T13:07:14.086 INFO:teuthology.orchestra.run.smithi066.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T13:07:14.133 INFO:teuthology.orchestra.run.smithi031.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 195 kB/s | 26 kB 00:00 2024-04-25T13:07:14.158 INFO:teuthology.orchestra.run.smithi031.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 364 kB/s | 49 kB 00:00 2024-04-25T13:07:14.186 INFO:teuthology.orchestra.run.smithi066.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 170 kB 00:00 2024-04-25T13:07:14.217 INFO:teuthology.orchestra.run.smithi031.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 807 kB/s | 67 kB 00:00 2024-04-25T13:07:14.268 INFO:teuthology.orchestra.run.smithi136.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 1.3 MB 00:00 2024-04-25T13:07:14.278 INFO:teuthology.orchestra.run.smithi066.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 373 kB/s | 34 kB 00:00 2024-04-25T13:07:14.369 INFO:teuthology.orchestra.run.smithi066.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.5 MB/s | 142 kB 00:00 2024-04-25T13:07:14.442 INFO:teuthology.orchestra.run.smithi031.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 533 kB/s | 151 kB 00:00 2024-04-25T13:07:14.478 INFO:teuthology.orchestra.run.smithi066.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 7.2 MB/s | 3.1 MB 00:00 2024-04-25T13:07:14.718 INFO:teuthology.orchestra.run.smithi031.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.6 MB/s | 838 kB 00:00 2024-04-25T13:07:14.776 INFO:teuthology.orchestra.run.smithi031.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 1.6 MB/s | 548 kB 00:00 2024-04-25T13:07:14.802 INFO:teuthology.orchestra.run.smithi031.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-04-25T13:07:14.829 INFO:teuthology.orchestra.run.smithi066.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.7 MB/s | 3.5 MB 00:00 2024-04-25T13:07:14.860 INFO:teuthology.orchestra.run.smithi031.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 384 kB/s | 32 kB 00:00 2024-04-25T13:07:14.886 INFO:teuthology.orchestra.run.smithi031.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-25T13:07:14.912 INFO:teuthology.orchestra.run.smithi066.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.9 MB/s | 249 kB 00:00 2024-04-25T13:07:14.977 INFO:teuthology.orchestra.run.smithi031.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-25T13:07:14.996 INFO:teuthology.orchestra.run.smithi066.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 591 kB/s | 49 kB 00:00 2024-04-25T13:07:15.071 INFO:teuthology.orchestra.run.smithi066.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 220 kB/s | 16 kB 00:00 2024-04-25T13:07:15.103 INFO:teuthology.orchestra.run.smithi031.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-25T13:07:15.128 INFO:teuthology.orchestra.run.smithi031.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-04-25T13:07:15.154 INFO:teuthology.orchestra.run.smithi066.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 273 kB 00:00 2024-04-25T13:07:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-04-25T13:07:15.262 INFO:teuthology.orchestra.run.smithi031.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-04-25T13:07:15.313 INFO:teuthology.orchestra.run.smithi066.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 4.7 MB/s | 762 kB 00:00 2024-04-25T13:07:15.404 INFO:teuthology.orchestra.run.smithi031.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-25T13:07:15.430 INFO:teuthology.orchestra.run.smithi066.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 7.8 MB/s | 7.4 MB 00:00 2024-04-25T13:07:15.456 INFO:teuthology.orchestra.run.smithi066.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 128 kB/s | 18 kB 00:00 2024-04-25T13:07:15.581 INFO:teuthology.orchestra.run.smithi066.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-04-25T13:07:15.596 INFO:teuthology.orchestra.run.smithi031.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-04-25T13:07:15.639 INFO:teuthology.orchestra.run.smithi066.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-04-25T13:07:15.665 INFO:teuthology.orchestra.run.smithi066.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-25T13:07:15.690 INFO:teuthology.orchestra.run.smithi066.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T13:07:15.713 INFO:teuthology.orchestra.run.smithi031.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2024-04-25T13:07:15.735 INFO:teuthology.orchestra.run.smithi136.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.4 MB/s | 2.4 MB 00:01 2024-04-25T13:07:15.740 INFO:teuthology.orchestra.run.smithi066.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-25T13:07:15.763 INFO:teuthology.orchestra.run.smithi031.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-04-25T13:07:15.765 INFO:teuthology.orchestra.run.smithi066.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-25T13:07:15.791 INFO:teuthology.orchestra.run.smithi066.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T13:07:15.813 INFO:teuthology.orchestra.run.smithi031.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-25T13:07:15.816 INFO:teuthology.orchestra.run.smithi066.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T13:07:15.841 INFO:teuthology.orchestra.run.smithi066.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T13:07:15.855 INFO:teuthology.orchestra.run.smithi031.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-25T13:07:15.867 INFO:teuthology.orchestra.run.smithi066.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-25T13:07:15.900 INFO:teuthology.orchestra.run.smithi066.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T13:07:15.906 INFO:teuthology.orchestra.run.smithi031.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-25T13:07:15.926 INFO:teuthology.orchestra.run.smithi066.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T13:07:15.943 INFO:teuthology.orchestra.run.smithi066.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 514 kB/s | 21 kB 00:00 2024-04-25T13:07:15.968 INFO:teuthology.orchestra.run.smithi066.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-25T13:07:15.985 INFO:teuthology.orchestra.run.smithi066.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 793 kB/s | 33 kB 00:00 2024-04-25T13:07:15.989 INFO:teuthology.orchestra.run.smithi031.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 557 kB/s | 46 kB 00:00 2024-04-25T13:07:16.081 INFO:teuthology.orchestra.run.smithi031.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 861 kB/s | 79 kB 00:00 2024-04-25T13:07:16.085 INFO:teuthology.orchestra.run.smithi066.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-25T13:07:16.102 INFO:teuthology.orchestra.run.smithi066.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-04-25T13:07:16.119 INFO:teuthology.orchestra.run.smithi066.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 334 kB/s | 11 kB 00:00 2024-04-25T13:07:16.144 INFO:teuthology.orchestra.run.smithi066.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-25T13:07:16.169 INFO:teuthology.orchestra.run.smithi066.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-04-25T13:07:16.181 INFO:teuthology.orchestra.run.smithi031.stdout:(131/152): libarrow-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 4.4 MB 00:02 2024-04-25T13:07:16.195 INFO:teuthology.orchestra.run.smithi066.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T13:07:16.220 INFO:teuthology.orchestra.run.smithi066.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 582 kB 00:00 2024-04-25T13:07:16.231 INFO:teuthology.orchestra.run.smithi031.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 384 kB/s | 58 kB 00:00 2024-04-25T13:07:16.245 INFO:teuthology.orchestra.run.smithi066.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 515 kB/s | 25 kB 00:00 2024-04-25T13:07:16.265 INFO:teuthology.orchestra.run.smithi031.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 2.5 MB/s | 1.0 MB 00:00 2024-04-25T13:07:16.271 INFO:teuthology.orchestra.run.smithi066.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-25T13:07:16.296 INFO:teuthology.orchestra.run.smithi066.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T13:07:16.315 INFO:teuthology.orchestra.run.smithi031.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-25T13:07:16.321 INFO:teuthology.orchestra.run.smithi066.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T13:07:16.346 INFO:teuthology.orchestra.run.smithi066.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-04-25T13:07:16.349 INFO:teuthology.orchestra.run.smithi031.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-25T13:07:16.382 INFO:teuthology.orchestra.run.smithi031.stdout:(136/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-25T13:07:16.408 INFO:teuthology.orchestra.run.smithi031.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-25T13:07:16.449 INFO:teuthology.orchestra.run.smithi031.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-25T13:07:16.455 INFO:teuthology.orchestra.run.smithi066.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-04-25T13:07:16.492 INFO:teuthology.orchestra.run.smithi031.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-04-25T13:07:16.539 INFO:teuthology.orchestra.run.smithi066.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-04-25T13:07:16.550 INFO:teuthology.orchestra.run.smithi031.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-04-25T13:07:16.552 INFO:teuthology.orchestra.run.smithi136.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 3.8 MB/s | 49 MB 00:12 2024-04-25T13:07:16.559 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:07:16.559 INFO:teuthology.orchestra.run.smithi136.stdout:Total 14 MB/s | 222 MB 00:15 2024-04-25T13:07:16.575 INFO:teuthology.orchestra.run.smithi031.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 285 kB/s | 36 kB 00:00 2024-04-25T13:07:16.581 INFO:teuthology.orchestra.run.smithi066.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T13:07:16.600 INFO:teuthology.orchestra.run.smithi031.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-04-25T13:07:16.623 INFO:teuthology.orchestra.run.smithi066.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T13:07:16.648 INFO:teuthology.orchestra.run.smithi066.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T13:07:16.676 INFO:teuthology.orchestra.run.smithi031.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-04-25T13:07:16.740 INFO:teuthology.orchestra.run.smithi066.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-25T13:07:16.768 INFO:teuthology.orchestra.run.smithi031.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-04-25T13:07:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:(145/152): python3-webob-1.8.7-6.el9.noarch.rpm 947 kB/s | 230 kB 00:00 2024-04-25T13:07:16.815 INFO:teuthology.orchestra.run.smithi066.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-04-25T13:07:16.818 INFO:teuthology.orchestra.run.smithi031.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-04-25T13:07:16.841 INFO:teuthology.orchestra.run.smithi066.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 891 kB/s | 87 kB 00:00 2024-04-25T13:07:16.974 INFO:teuthology.orchestra.run.smithi066.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-04-25T13:07:16.999 INFO:teuthology.orchestra.run.smithi066.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-25T13:07:17.025 INFO:teuthology.orchestra.run.smithi066.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T13:07:17.044 INFO:teuthology.orchestra.run.smithi031.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 695 kB/s | 192 kB 00:00 2024-04-25T13:07:17.050 INFO:teuthology.orchestra.run.smithi066.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-04-25T13:07:17.075 INFO:teuthology.orchestra.run.smithi066.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-04-25T13:07:17.101 INFO:teuthology.orchestra.run.smithi066.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T13:07:17.126 INFO:teuthology.orchestra.run.smithi066.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 698 kB/s | 35 kB 00:00 2024-04-25T13:07:17.260 INFO:teuthology.orchestra.run.smithi066.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-25T13:07:17.285 INFO:teuthology.orchestra.run.smithi066.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-25T13:07:17.310 INFO:teuthology.orchestra.run.smithi066.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T13:07:17.335 INFO:teuthology.orchestra.run.smithi066.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T13:07:17.360 INFO:teuthology.orchestra.run.smithi066.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-04-25T13:07:17.386 INFO:teuthology.orchestra.run.smithi066.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-25T13:07:17.402 INFO:teuthology.orchestra.run.smithi031.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 5.8 MB/s | 3.4 MB 00:00 2024-04-25T13:07:17.411 INFO:teuthology.orchestra.run.smithi066.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T13:07:17.436 INFO:teuthology.orchestra.run.smithi066.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T13:07:17.452 INFO:teuthology.orchestra.run.smithi031.stdout:(149/152): thrift-0.15.0-2.el9.x86_64.rpm 2.4 MB/s | 1.6 MB 00:00 2024-04-25T13:07:17.453 INFO:teuthology.orchestra.run.smithi066.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T13:07:17.470 INFO:teuthology.orchestra.run.smithi066.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-25T13:07:17.594 INFO:teuthology.orchestra.run.smithi031.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 1.3 MB 00:00 2024-04-25T13:07:17.645 INFO:teuthology.orchestra.run.smithi031.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-04-25T13:07:17.670 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-25T13:07:17.703 INFO:teuthology.orchestra.run.smithi031.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 4.8 MB/s | 3.1 MB 00:00 2024-04-25T13:07:17.708 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:07:17.709 INFO:teuthology.orchestra.run.smithi031.stdout:Total 8.4 MB/s | 222 MB 00:26 2024-04-25T13:07:17.813 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-25T13:07:17.813 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-25T13:07:17.962 INFO:teuthology.orchestra.run.smithi066.stdout:(91/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 8.0 MB/s | 49 MB 00:06 2024-04-25T13:07:17.996 INFO:teuthology.orchestra.run.smithi066.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 79 kB/s | 42 kB 00:00 2024-04-25T13:07:18.255 INFO:teuthology.orchestra.run.smithi066.stdout:(93/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-25T13:07:18.271 INFO:teuthology.orchestra.run.smithi066.stdout:(94/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 145 kB/s | 44 kB 00:00 2024-04-25T13:07:18.297 INFO:teuthology.orchestra.run.smithi066.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 742 kB/s | 221 kB 00:00 2024-04-25T13:07:18.322 INFO:teuthology.orchestra.run.smithi066.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 539 kB/s | 36 kB 00:00 2024-04-25T13:07:18.347 INFO:teuthology.orchestra.run.smithi066.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-25T13:07:18.373 INFO:teuthology.orchestra.run.smithi066.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T13:07:18.398 INFO:teuthology.orchestra.run.smithi066.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 774 kB/s | 58 kB 00:00 2024-04-25T13:07:18.423 INFO:teuthology.orchestra.run.smithi066.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-25T13:07:18.448 INFO:teuthology.orchestra.run.smithi066.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 850 kB/s | 64 kB 00:00 2024-04-25T13:07:18.473 INFO:teuthology.orchestra.run.smithi066.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 597 kB/s | 45 kB 00:00 2024-04-25T13:07:18.499 INFO:teuthology.orchestra.run.smithi066.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-04-25T13:07:18.524 INFO:teuthology.orchestra.run.smithi066.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-25T13:07:18.691 INFO:teuthology.orchestra.run.smithi066.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 117 kB/s | 19 kB 00:00 2024-04-25T13:07:18.833 INFO:teuthology.orchestra.run.smithi066.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-04-25T13:07:18.860 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-25T13:07:18.916 INFO:teuthology.orchestra.run.smithi066.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-04-25T13:07:18.941 INFO:teuthology.orchestra.run.smithi066.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-25T13:07:18.983 INFO:teuthology.orchestra.run.smithi066.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 727 kB/s | 49 kB 00:00 2024-04-25T13:07:19.009 INFO:teuthology.orchestra.run.smithi066.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-25T13:07:19.019 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-25T13:07:19.019 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-25T13:07:19.059 INFO:teuthology.orchestra.run.smithi066.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-25T13:07:19.142 INFO:teuthology.orchestra.run.smithi066.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 838 kB 00:00 2024-04-25T13:07:19.218 INFO:teuthology.orchestra.run.smithi066.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-25T13:07:19.243 INFO:teuthology.orchestra.run.smithi066.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 433 kB/s | 43 kB 00:00 2024-04-25T13:07:19.302 INFO:teuthology.orchestra.run.smithi066.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-04-25T13:07:19.327 INFO:teuthology.orchestra.run.smithi066.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 180 kB/s | 15 kB 00:00 2024-04-25T13:07:19.394 INFO:teuthology.orchestra.run.smithi066.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-25T13:07:19.419 INFO:teuthology.orchestra.run.smithi066.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-25T13:07:19.503 INFO:teuthology.orchestra.run.smithi066.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-25T13:07:19.528 INFO:teuthology.orchestra.run.smithi066.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-04-25T13:07:19.620 INFO:teuthology.orchestra.run.smithi066.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T13:07:19.667 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-25T13:07:19.667 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-25T13:07:19.804 INFO:teuthology.orchestra.run.smithi066.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 788 kB/s | 144 kB 00:00 2024-04-25T13:07:19.846 INFO:teuthology.orchestra.run.smithi066.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-04-25T13:07:19.879 INFO:teuthology.orchestra.run.smithi066.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-25T13:07:19.913 INFO:teuthology.orchestra.run.smithi066.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-25T13:07:19.947 INFO:teuthology.orchestra.run.smithi066.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-04-25T13:07:19.980 INFO:teuthology.orchestra.run.smithi066.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 289 kB/s | 19 kB 00:00 2024-04-25T13:07:20.014 INFO:teuthology.orchestra.run.smithi066.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 292 kB/s | 20 kB 00:00 2024-04-25T13:07:20.064 INFO:teuthology.orchestra.run.smithi066.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-25T13:07:20.165 INFO:teuthology.orchestra.run.smithi066.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 308 kB/s | 46 kB 00:00 2024-04-25T13:07:20.190 INFO:teuthology.orchestra.run.smithi066.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 628 kB/s | 79 kB 00:00 2024-04-25T13:07:20.223 INFO:teuthology.orchestra.run.smithi066.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 989 kB/s | 58 kB 00:00 2024-04-25T13:07:20.265 INFO:teuthology.orchestra.run.smithi066.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T13:07:20.282 INFO:teuthology.orchestra.run.smithi066.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-04-25T13:07:20.332 INFO:teuthology.orchestra.run.smithi066.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-25T13:07:20.358 INFO:teuthology.orchestra.run.smithi066.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-25T13:07:20.408 INFO:teuthology.orchestra.run.smithi066.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-25T13:07:20.433 INFO:teuthology.orchestra.run.smithi066.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-25T13:07:20.484 INFO:teuthology.orchestra.run.smithi066.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-04-25T13:07:20.509 INFO:teuthology.orchestra.run.smithi066.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-25T13:07:20.551 INFO:teuthology.orchestra.run.smithi066.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 762 kB/s | 51 kB 00:00 2024-04-25T13:07:20.584 INFO:teuthology.orchestra.run.smithi066.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T13:07:20.626 INFO:teuthology.orchestra.run.smithi066.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T13:07:20.693 INFO:teuthology.orchestra.run.smithi066.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-04-25T13:07:20.727 INFO:teuthology.orchestra.run.smithi066.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-25T13:07:20.769 INFO:teuthology.orchestra.run.smithi066.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T13:07:20.899 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-25T13:07:20.900 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-25T13:07:20.952 INFO:teuthology.orchestra.run.smithi066.stdout:(147/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.0 MB/s | 4.4 MB 00:02 2024-04-25T13:07:21.153 INFO:teuthology.orchestra.run.smithi066.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-04-25T13:07:21.253 INFO:teuthology.orchestra.run.smithi066.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 14 MB/s | 1.3 MB 00:00 2024-04-25T13:07:21.453 INFO:teuthology.orchestra.run.smithi066.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-04-25T13:07:21.679 INFO:teuthology.orchestra.run.smithi066.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 3.7 MB/s | 3.4 MB 00:00 2024-04-25T13:07:21.770 INFO:teuthology.orchestra.run.smithi066.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 3.8 MB/s | 3.1 MB 00:00 2024-04-25T13:07:21.775 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:07:21.776 INFO:teuthology.orchestra.run.smithi066.stdout:Total 11 MB/s | 222 MB 00:20 2024-04-25T13:07:22.941 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-04-25T13:07:23.101 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-04-25T13:07:23.101 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-04-25T13:07:24.976 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-04-25T13:07:24.976 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-04-25T13:07:25.832 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-25T13:07:25.913 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:07:25.951 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:07:26.097 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:07:26.139 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:07:26.164 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:07:26.207 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:07:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:07:26.640 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:07:26.762 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:07:26.792 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:07:26.883 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:07:26.912 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:07:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:07:27.070 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-25T13:07:27.109 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:07:27.137 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:07:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:07:27.212 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:07:27.257 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:07:27.266 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:07:27.288 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:07:27.299 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:07:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:07:27.343 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:07:27.364 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:07:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:07:27.387 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:07:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:07:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:07:27.688 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:07:27.712 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:07:27.784 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:07:27.825 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:07:27.830 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:07:27.865 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:07:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:07:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:07:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:07:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:07:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:07:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:07:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:07:28.069 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:07:28.122 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:07:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:07:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:07:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:07:28.274 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:07:28.279 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:07:28.336 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:07:28.351 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:07:28.363 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:07:28.404 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T13:07:28.445 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T13:07:28.452 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:07:28.473 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T13:07:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:07:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:07:28.543 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:07:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:07:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:07:28.588 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:07:28.662 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:07:28.725 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:07:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:07:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:07:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:07:28.866 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:07:28.963 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:07:28.997 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:07:29.032 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:07:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:07:29.148 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:07:29.191 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:07:29.223 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:07:29.251 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:07:29.323 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:07:29.375 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:07:29.385 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:07:29.404 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:07:29.448 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:07:29.469 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:07:29.498 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:07:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:07:29.541 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:07:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T13:07:29.579 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T13:07:29.610 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T13:07:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:07:29.647 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T13:07:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:07:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:07:29.810 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:07:29.840 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:07:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:07:29.937 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:07:30.009 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:07:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:07:30.049 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:07:30.086 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:07:30.132 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:07:30.169 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:07:30.212 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:07:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:07:30.390 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:07:30.451 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:07:30.481 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:07:30.523 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:07:30.604 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:07:30.616 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:07:30.665 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:07:30.705 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:07:30.730 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:07:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:07:30.811 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T13:07:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:07:30.853 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:07:30.959 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:07:30.996 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:07:31.008 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:07:31.024 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:07:31.058 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:07:31.063 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:07:31.097 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:07:31.119 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:07:31.129 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:07:31.159 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:07:31.183 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-04-25T13:07:31.193 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:07:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:07:31.227 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:07:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:07:31.258 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:07:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:07:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:07:31.297 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:07:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:07:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:07:31.351 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:07:31.384 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:07:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:07:31.409 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:07:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:07:31.519 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:07:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:07:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:07:31.646 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:07:31.649 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:07:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:07:31.697 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:07:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:07:31.806 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:07:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:07:31.826 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:07:31.858 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:07:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:07:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:07:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:07:31.915 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:07:31.987 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:07:32.043 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:07:32.069 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:07:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:07:32.150 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:07:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:07:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:07:32.199 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:07:32.212 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:07:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:07:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:07:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:07:32.359 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:07:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:07:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:07:32.406 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:07:32.434 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:07:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:07:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:07:32.539 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:07:32.576 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:07:32.577 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:07:32.577 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:07:32.622 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:07:32.631 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:07:32.653 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:07:32.655 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:07:32.676 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:07:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:07:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:07:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:07:32.820 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:07:32.853 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:07:32.870 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:07:32.886 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:07:32.893 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:07:32.933 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:07:32.997 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:07:33.093 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:07:33.120 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:07:33.134 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:07:33.147 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:07:33.170 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:07:33.226 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:07:33.247 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:07:33.285 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:07:33.329 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:07:33.361 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:07:33.389 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:07:33.462 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:07:33.512 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:07:33.541 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:07:33.577 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:07:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:07:33.655 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:07:33.687 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T13:07:33.691 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:07:33.725 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:07:33.731 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T13:07:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T13:07:33.828 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:07:33.834 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:07:33.879 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:07:33.884 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:07:33.891 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:07:33.970 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:07:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:07:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:07:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:07:34.123 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:07:34.194 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:07:34.201 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:07:34.232 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:07:34.329 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:07:34.753 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:07:34.842 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:07:34.876 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:07:34.919 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:07:34.957 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T13:07:34.999 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:07:35.013 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:07:35.126 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:07:35.169 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:07:35.215 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:07:35.226 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:07:35.283 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:07:35.286 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:07:35.313 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:07:35.339 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:07:35.357 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:07:35.393 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:07:35.430 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:07:35.478 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:07:35.512 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:07:35.557 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:07:35.598 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:07:35.652 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:07:35.693 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:07:35.744 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:07:35.784 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:07:35.787 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:07:35.815 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T13:07:35.816 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:07:35.850 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:07:35.859 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:07:35.958 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:07:36.000 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:07:36.040 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:07:36.065 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:07:36.089 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:07:36.168 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:07:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:07:36.330 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:07:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:07:36.358 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:07:36.363 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:07:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:07:36.430 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:07:36.432 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:07:36.434 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:07:36.465 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T13:07:36.472 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:07:36.501 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:07:36.510 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:07:36.536 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:07:36.538 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:07:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:07:36.571 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:07:36.678 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:07:36.711 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:07:36.802 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:07:36.851 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:07:36.882 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:07:36.899 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:07:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:07:36.933 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:07:37.025 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:07:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:07:37.055 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T13:07:37.091 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:07:37.093 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:07:37.588 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:07:37.620 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:07:37.691 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:07:37.713 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T13:07:37.863 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:07:37.913 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:07:37.947 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:07:38.054 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:07:38.115 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:07:38.207 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:07:38.241 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:07:38.426 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:07:38.455 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:07:38.535 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:07:38.795 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:07:38.836 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:07:39.225 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:07:39.349 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:07:39.383 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:07:39.423 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:07:39.520 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:07:39.520 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:07:39.520 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:07:39.520 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:39.550 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:07:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:07:40.102 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:07:40.667 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:07:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:07:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:07:40.749 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:07:40.778 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:07:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:07:40.786 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:07:40.787 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:07:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:07:40.810 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:07:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:07:41.139 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:07:41.181 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:07:41.273 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:07:41.303 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T13:07:41.340 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:07:41.845 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:07:41.877 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:07:41.948 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:07:41.978 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T13:07:44.419 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:07:44.458 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:07:45.031 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:07:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:07:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:07:45.170 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:07:45.170 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:07:45.170 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:07:45.204 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:07:51.015 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:07:51.015 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2024-04-25T13:07:51.016 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2024-04-25T13:07:51.016 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2024-04-25T13:07:51.016 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2024-04-25T13:07:51.016 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2024-04-25T13:07:51.016 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /root 2024-04-25T13:07:51.016 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2024-04-25T13:07:51.016 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:07:51.203 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:07:51.203 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:51.203 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:07:51.203 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:52.690 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:07:52.692 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:07:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-04-25T13:07:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-04-25T13:07:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-04-25T13:07:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-04-25T13:07:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-04-25T13:07:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-04-25T13:07:52.692 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-04-25T13:07:52.693 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:07:52.716 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:07:52.716 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:52.716 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:07:52.716 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:07:52.716 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:07:52.716 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:07:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:07:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:07:52.973 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:52.973 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:07:52.973 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:07:52.974 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:07:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:53.418 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:07:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:07:53.440 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:53.440 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:07:53.441 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:07:53.441 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:07:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:07:53.587 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:07:53.643 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:07:53.826 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:07:53.877 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:07:53.945 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:07:54.034 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:07:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:07:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:07:54.242 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:07:54.271 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:07:54.418 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:07:54.445 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:07:54.445 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:54.445 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:07:54.445 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:07:54.445 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:07:54.445 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:07:54.677 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:07:54.701 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:07:54.701 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:54.701 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:07:54.701 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:07:54.701 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:07:54.701 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:07:54.714 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:07:54.745 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:07:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T13:07:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T13:07:54.877 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:07:55.169 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:07:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:07:55.193 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:55.193 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:07:55.193 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:07:55.193 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:07:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:07:55.300 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:07:55.357 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:07:55.421 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:07:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:07:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:07:55.752 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:07:55.849 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:07:55.910 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:07:55.945 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:07:55.999 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:07:56.036 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:07:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:07:56.095 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:07:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:07:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:07:56.619 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T13:07:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T13:07:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:07:56.887 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:07:56.887 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2024-04-25T13:07:56.887 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2024-04-25T13:07:56.888 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2024-04-25T13:07:56.888 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2024-04-25T13:07:56.888 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2024-04-25T13:07:56.888 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /root 2024-04-25T13:07:56.888 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2024-04-25T13:07:56.888 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:07:56.903 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:07:56.928 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:07:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:07:57.066 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:07:57.082 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:07:57.082 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:57.082 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:07:57.083 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:07:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T13:07:57.222 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:07:57.246 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:07:57.246 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:57.246 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:07:57.246 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:07:57.246 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:07:57.246 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:57.290 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:07:57.301 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:07:57.398 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T13:07:57.446 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T13:07:57.811 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:07:57.843 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:07:58.368 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:07:58.401 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:07:58.424 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:07:58.424 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:58.424 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:07:58.424 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:07:58.424 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:07:58.424 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:58.468 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:07:58.491 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:07:58.491 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:58.491 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:07:58.491 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:58.536 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:07:58.558 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:07:58.558 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:58.558 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:07:58.558 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:07:58.558 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:07:58.558 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:58.619 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:07:58.646 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:07:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:07:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:07:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:07:58.647 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:07:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:07:58.861 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:07:58.878 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:07:58.884 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:07:58.884 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:58.884 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:07:58.884 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:07:58.884 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:07:58.884 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:07:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:07:58.902 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:07:58.902 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:58.902 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:07:58.902 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:07:58.902 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:07:58.902 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:07:58.994 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:07:59.164 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T13:07:59.187 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:07:59.370 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:07:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T13:07:59.395 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:07:59.395 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:07:59.395 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:07:59.395 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:07:59.395 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:07:59.395 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:07:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T13:07:59.509 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:07:59.567 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:07:59.632 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:07:59.827 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:07:59.888 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:07:59.969 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:08:00.067 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:08:00.219 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:08:00.263 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:08:00.295 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:08:00.322 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:08:00.403 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:08:00.431 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:08:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:08:00.456 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:08:00.456 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:08:00.456 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:08:00.456 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:08:00.456 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:08:00.499 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:08:00.522 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:08:00.522 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:08:00.522 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:08:00.523 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:08:00.574 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:08:00.598 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:08:00.598 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:08:00.598 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:08:00.598 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:08:00.598 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:08:00.598 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:08:00.792 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:08:00.821 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:08:00.853 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T13:08:00.917 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T13:08:00.925 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:08:00.949 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:08:00.949 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:08:00.949 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:08:00.949 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:08:00.950 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:08:00.950 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:08:00.961 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:08:02.059 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:08:02.096 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:08:03.112 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:08:03.137 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:08:03.247 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:08:03.408 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T13:08:03.431 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:08:03.456 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:08:03.457 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:08:03.457 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:08:03.457 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:08:03.457 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:08:03.457 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:08:03.481 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T13:08:03.507 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:08:03.519 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:08:03.523 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T13:08:03.566 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T13:08:03.596 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:08:03.620 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T13:08:03.631 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:08:03.674 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T13:08:03.816 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T13:08:03.921 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:08:03.921 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T13:08:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:08:04.539 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:08:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:08:04.648 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:08:04.684 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:08:04.709 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:08:04.709 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:08:04.709 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:08:04.709 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:08:04.709 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:08:04.709 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:08:04.751 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:08:04.775 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:08:04.775 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:08:04.775 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:08:04.775 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:08:04.832 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:08:04.856 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:08:04.856 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:08:04.856 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:08:04.856 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:08:04.856 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:08:04.856 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:08:05.170 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:08:05.194 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:08:05.194 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:08:05.194 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:08:05.194 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:08:05.197 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:08:05.197 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:08:05.395 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T13:08:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:08:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:08:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:08:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:08:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:08:05.759 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:08:05.792 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:08:05.792 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:08:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T13:08:05.836 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T13:08:05.877 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T13:08:05.898 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:08:05.909 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:08:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:08:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:08:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T13:08:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:08:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T13:08:06.846 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:08:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:08:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:08:07.825 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T13:08:07.915 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:08:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:08:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:08:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:08:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:08:08.170 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:08:08.211 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:08:08.211 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:08:08.302 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:08:10.049 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T13:08:10.090 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T13:08:10.131 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T13:08:10.163 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:08:10.190 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:08:10.385 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T13:08:10.427 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:08:10.448 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T13:08:11.086 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:08:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:08:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:08:12.060 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T13:08:12.159 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:08:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:08:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T13:08:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T13:08:12.177 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T13:08:12.178 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T13:08:12.179 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:08:12.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:08:12.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:08:12.182 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:08:12.183 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:08:12.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:08:12.185 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:08:12.186 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T13:08:12.187 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:08:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T13:08:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:08:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:08:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:08:12.188 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:08:12.201 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:08:12.286 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:08:12.321 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:08:12.321 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:08:12.406 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:08:12.438 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:08:12.438 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:08:12.529 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:08:12.598 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:08:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:08:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T13:08:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T13:08:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T13:08:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T13:08:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T13:08:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T13:08:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T13:08:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:08:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:08:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:08:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T13:08:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T13:08:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T13:08:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T13:08:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T13:08:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T13:08:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T13:08:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T13:08:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T13:08:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T13:08:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T13:08:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T13:08:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T13:08:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T13:08:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T13:08:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T13:08:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T13:08:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T13:08:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:08:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:08:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:08:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:08:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:08:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:08:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:08:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:08:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:08:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:08:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:08:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:08:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:08:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:08:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:08:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:08:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:08:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:08:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:08:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:08:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:08:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:08:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:08:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:08:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:08:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:08:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:08:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:08:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:08:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:08:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:08:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:08:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:08:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:08:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:08:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:08:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:08:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:08:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:08:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:08:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:08:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:08:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:08:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:08:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:08:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:08:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:08:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:08:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:08:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:08:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:08:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:08:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:08:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:08:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T13:08:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:08:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T13:08:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:08:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:08:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:08:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:08:14.269 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:08:14.269 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:08:14.269 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2024-04-25T13:08:14.269 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.269 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.269 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:08:14.269 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:08:14.269 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-25T13:08:14.269 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.271 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.272 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:08:14.273 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:08:14.274 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:08:14.275 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:08:14.276 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:08:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:08:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.280 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:14.280 INFO:teuthology.orchestra.run.smithi136.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:08:14.280 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:08:14.280 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:08:14.280 INFO:teuthology.orchestra.run.smithi136.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:08:14.280 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:08:14.280 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:08:14.280 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:08:14.280 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:08:14.280 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-25T13:08:14.664 DEBUG:teuthology.parallel:result is None 2024-04-25T13:08:16.025 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:08:16.026 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:08:16.026 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-04-25T13:08:16.026 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.026 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.026 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:08:16.026 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:08:16.026 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-25T13:08:16.026 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:08:16.026 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:08:16.026 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:16.027 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:16.028 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:16.028 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.028 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.028 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:16.028 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.028 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.028 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.028 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:16.028 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:16.029 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.029 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.029 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:16.029 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:08:16.029 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:08:16.029 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:08:16.029 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:08:16.029 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:08:16.030 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:08:16.031 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:08:16.032 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:08:16.033 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:08:16.034 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:08:16.035 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:08:16.036 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:08:16.037 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:08:16.037 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:08:16.037 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:08:16.037 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:08:16.037 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:08:16.037 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:08:16.037 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:08:16.037 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.037 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.037 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:16.038 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:08:16.038 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:08:16.038 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:08:16.038 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:08:16.038 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:08:16.038 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:08:16.038 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:08:16.038 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:08:16.038 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-25T13:08:16.409 DEBUG:teuthology.parallel:result is None 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T13:08:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T13:08:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T13:08:18.183 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T13:08:18.183 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T13:08:18.183 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T13:08:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T13:08:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T13:08:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T13:08:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:08:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:08:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:08:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:08:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:08:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:08:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:08:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:08:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:08:20.285 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:08:20.285 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:08:20.285 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2024-04-25T13:08:20.285 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.285 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.285 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:08:20.285 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:08:20.285 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-04-25T13:08:20.285 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:08:20.285 INFO:teuthology.orchestra.run.smithi066.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:20.286 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:20.287 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.288 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:08:20.289 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:08:20.290 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:08:20.291 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:08:20.292 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:08:20.293 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.294 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:08:20.295 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.296 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.296 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:08:20.296 INFO:teuthology.orchestra.run.smithi066.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:08:20.296 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:08:20.296 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:08:20.296 INFO:teuthology.orchestra.run.smithi066.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:08:20.296 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:08:20.296 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:08:20.296 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:08:20.296 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-04-25T13:08:20.296 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-04-25T13:08:20.671 DEBUG:teuthology.parallel:result is None 2024-04-25T13:08:20.672 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:08:20.826 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:08:20.852 INFO:teuthology.orchestra.run.smithi031.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T13:08:20.853 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T13:08:20.853 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T13:08:20.855 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:08:20.996 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:08:21.019 INFO:teuthology.orchestra.run.smithi066.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T13:08:21.019 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T13:08:21.019 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T13:08:21.021 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:08:21.253 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:08:21.278 INFO:teuthology.orchestra.run.smithi136.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T13:08:21.278 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T13:08:21.278 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T13:08:21.280 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T13:08:21.280 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:08:21.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:08:21.312 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-25T13:08:21.312 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:08:21.338 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T13:08:21.339 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:08:21.371 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T13:08:21.372 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:08:21.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:08:21.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:08:21.471 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-25T13:08:21.471 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:08:21.495 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:08:21.559 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T13:08:21.559 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:08:21.587 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:08:21.654 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T13:08:21.654 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:08:21.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:08:21.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:08:21.752 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-25T13:08:21.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:08:21.778 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:08:21.843 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T13:08:21.843 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:08:21.871 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:08:21.938 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T13:08:21.939 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:08:21.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:08:21.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:08:22.036 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-25T13:08:22.036 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:08:22.061 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:08:22.128 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T13:08:22.128 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:08:22.156 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:08:22.222 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T13:08:22.326 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T13:08:22.327 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:08:22.327 INFO:tasks.cephadm:Cluster fsid is e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:08:22.327 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T13:08:22.328 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.31', 'mon.b': '172.21.15.66', 'mon.c': '172.21.15.136'} 2024-04-25T13:08:22.328 INFO:tasks.cephadm:First mon is mon.a on smithi031 2024-04-25T13:08:22.328 INFO:tasks.cephadm:First mgr is x 2024-04-25T13:08:22.328 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T13:08:22.328 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2024-04-25T13:08:22.355 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname $(hostname -s) 2024-04-25T13:08:22.379 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname $(hostname -s) 2024-04-25T13:08:22.406 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T13:08:22.407 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:08:22.738 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T13:08:23.017 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:08:23.019 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T13:08:23.019 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:08:23.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 13:08 /home/ubuntu/cephtest/cephadm 2024-04-25T13:08:23.619 DEBUG:teuthology.orchestra.run.smithi066:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:08:24.037 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 13:08 /home/ubuntu/cephtest/cephadm 2024-04-25T13:08:24.037 DEBUG:teuthology.orchestra.run.smithi136:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:08:24.458 INFO:teuthology.orchestra.run.smithi136.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 13:08 /home/ubuntu/cephtest/cephadm 2024-04-25T13:08:24.458 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:08:24.478 DEBUG:teuthology.orchestra.run.smithi066:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:08:24.497 DEBUG:teuthology.orchestra.run.smithi136:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:08:24.538 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T13:08:24.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T13:08:24.542 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T13:08:24.548 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T13:08:26.092 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:08:26.149 INFO:teuthology.orchestra.run.smithi136.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:08:26.150 INFO:teuthology.orchestra.run.smithi066.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:08:56.932 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-25T13:08:56.933 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T13:08:56.933 INFO:teuthology.orchestra.run.smithi136.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T13:08:56.933 INFO:teuthology.orchestra.run.smithi136.stdout: "repo_digests": [ 2024-04-25T13:08:56.933 INFO:teuthology.orchestra.run.smithi136.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T13:08:56.933 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-25T13:08:56.933 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-25T13:09:24.910 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-25T13:09:24.911 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T13:09:24.911 INFO:teuthology.orchestra.run.smithi066.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T13:09:24.911 INFO:teuthology.orchestra.run.smithi066.stdout: "repo_digests": [ 2024-04-25T13:09:24.911 INFO:teuthology.orchestra.run.smithi066.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T13:09:24.911 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-04-25T13:09:24.911 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-25T13:09:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T13:09:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T13:09:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T13:09:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2024-04-25T13:09:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T13:09:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:09:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T13:09:25.518 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2024-04-25T13:09:25.581 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph 2024-04-25T13:09:25.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph 2024-04-25T13:09:25.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2024-04-25T13:09:25.670 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /etc/ceph 2024-04-25T13:09:25.698 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /etc/ceph 2024-04-25T13:09:25.726 INFO:tasks.cephadm:Writing seed config... 2024-04-25T13:09:25.727 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T13:09:25.727 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T13:09:25.727 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T13:09:25.727 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T13:09:25.727 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T13:09:25.727 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T13:09:25.728 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T13:09:25.729 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T13:09:25.730 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T13:09:25.731 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:09:25.731 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T13:09:25.746 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e4b023f4-0304-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T13:09:25.747 DEBUG:teuthology.orchestra.run.smithi031:mon.a> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T13:09:25.788 DEBUG:teuthology.orchestra.run.smithi031:mgr.x> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T13:09:25.830 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T13:09:25.831 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid e4b023f4-0304-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:09:26.027 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:09:26.027 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'e4b023f4-0304-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2024-04-25T13:09:26.028 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T13:09:26.028 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2024-04-25T13:09:26.065 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T13:09:26.065 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2024-04-25T13:09:26.065 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2024-04-25T13:09:26.072 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T13:09:26.073 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T13:09:26.078 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T13:09:26.079 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T13:09:26.085 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-04-25T13:09:26.091 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-04-25T13:09:26.091 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-04-25T13:09:26.092 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2024-04-25T13:09:26.115 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T13:09:26.115 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T13:09:26.115 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2024-04-25T13:09:26.115 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2024-04-25T13:09:26.122 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T13:09:26.122 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T13:09:26.128 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T13:09:26.128 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T13:09:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-04-25T13:09:26.141 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-04-25T13:09:26.141 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-04-25T13:09:26.141 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2024-04-25T13:09:26.141 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:09:26.141 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 140433500384416 on /run/cephadm/e4b023f4-0304-11ef-bc93-c7b262605968.lock 2024-04-25T13:09:26.141 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 140433500384416 acquired on /run/cephadm/e4b023f4-0304-11ef-bc93-c7b262605968.lock 2024-04-25T13:09:26.142 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2024-04-25T13:09:26.142 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2024-04-25T13:09:26.142 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-04-25T13:09:26.145 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.31 metric 100 2024-04-25T13:09:26.146 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2024-04-25T13:09:26.148 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T13:09:26.148 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T13:09:26.148 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T13:09:26.151 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T13:09:26.151 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T13:09:26.151 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T13:09:26.151 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T13:09:26.151 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-04-25T13:09:26.151 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T13:09:26.151 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-04-25T13:09:26.152 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-04-25T13:09:26.152 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T13:09:26.152 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T13:09:26.153 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:09:26.646 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T13:09:26.646 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:09:26.646 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T13:09:26.646 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T13:09:26.646 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T13:09:26.646 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T13:09:26.646 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T13:09:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T13:09:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T13:09:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T13:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2024-04-25T13:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2024-04-25T13:09:29.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQAIVipm2cPSMhAAbvgWUQVDx/XHzqDktb0G0Q== 2024-04-25T13:09:30.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQAJVipmugdNKhAAGtT9fJCduDWcPwrBIUlhFw== 2024-04-25T13:09:31.212 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQAKVipmGwDZIhAA7uAq1/gWwsJ7rQ6IKm4HuA== 2024-04-25T13:09:31.213 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2024-04-25T13:09:32.144 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T13:09:32.144 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T13:09:32.144 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:09:32.144 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T13:09:32.144 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for a [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T13:09:32.144 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-04-25T13:09:32.144 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:09:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T13:09:32.145 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:09:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2024-04-25T13:09:33.042 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.a on 2024-04-25T13:09:33.264 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T13:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T13:09:33.612 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e4b023f4-0304-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-e4b023f4-0304-11ef-bc93-c7b262605968.target. 2024-04-25T13:09:33.612 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e4b023f4-0304-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-e4b023f4-0304-11ef-bc93-c7b262605968.target. 2024-04-25T13:09:33.837 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.a 2024-04-25T13:09:33.837 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T13:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e4b023f4-0304-11ef-bc93-c7b262605968.target.wants/ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-e4b023f4-0304-11ef-bc93-c7b262605968@.service. 2024-04-25T13:09:34.527 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:34 smithi031 ceph-mon[29083]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:09:34.541 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:09:34.541 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T13:09:34.548 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:09:34.548 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T13:09:34.548 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T13:09:34.548 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T13:09:34.549 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2024-04-25T13:09:34.549 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2024-04-25T13:09:34.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:34 smithi031 bash[29063]: ab6114c1a95941cfcefe509435d66a8860368f88a13055c670f7c0c3949bed37 2024-04-25T13:09:34.849 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:34 smithi031 systemd[1]: Started Ceph mon.a for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:09:35.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T13:09:35.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:09:35.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T13:09:35.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T13:09:35.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2024-04-25T13:09:35.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.542726s) 2024-04-25T13:09:35.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T13:09:35.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T13:09:35.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T13:09:35.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2024-04-25T13:09:35.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.950+0000 7f6a4d1e8700 1 Processor -- start 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.951+0000 7f6a4d1e8700 1 -- start start 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.951+0000 7f6a4d1e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a48108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.951+0000 7f6a4d1e8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48108ff0 con 0x7f6a48108600 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.951+0000 7f6a46d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a48108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.951+0000 7f6a46d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a48108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45760/0 (socket says 172.21.15.31:45760) 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.951+0000 7f6a46d9d700 1 -- 172.21.15.31:0/1452030304 learned_addr learned my addr 172.21.15.31:0/1452030304 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.951+0000 7f6a46d9d700 1 -- 172.21.15.31:0/1452030304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a481097b0 con 0x7f6a48108600 2024-04-25T13:09:35.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.952+0000 7f6a46d9d700 1 --2- 172.21.15.31:0/1452030304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a48108a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0098a0 tx=0x7f6a3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ead326ebf0618b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.952+0000 7f6a45d9b700 1 -- 172.21.15.31:0/1452030304 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a3c004160 con 0x7f6a48108600 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.952+0000 7f6a45d9b700 1 -- 172.21.15.31:0/1452030304 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c004300 con 0x7f6a48108600 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.952+0000 7f6a45d9b700 1 -- 172.21.15.31:0/1452030304 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a3c0036a0 con 0x7f6a48108600 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.952+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/1452030304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 msgr2=0x7f6a48108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.952+0000 7f6a4d1e8700 1 --2- 172.21.15.31:0/1452030304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a48108a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0098a0 tx=0x7f6a3c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.952+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/1452030304 shutdown_connections 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.952+0000 7f6a4d1e8700 1 --2- 172.21.15.31:0/1452030304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a48108a20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.952+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/1452030304 >> 172.21.15.31:0/1452030304 conn(0x7f6a48103bc0 msgr2=0x7f6a48105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.952+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/1452030304 shutdown_connections 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.952+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/1452030304 wait complete. 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.953+0000 7f6a4d1e8700 1 Processor -- start 2024-04-25T13:09:35.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.953+0000 7f6a4d1e8700 1 -- start start 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a4d1e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a4807db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a4d1e8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48109ce0 con 0x7f6a48108600 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a46d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a4807db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a46d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a4807db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45766/0 (socket says 172.21.15.31:45766) 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a46d9d700 1 -- 172.21.15.31:0/3511775229 learned_addr learned my addr 172.21.15.31:0/3511775229 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a46d9d700 1 -- 172.21.15.31:0/3511775229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4807e090 con 0x7f6a48108600 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a46d9d700 1 --2- 172.21.15.31:0/3511775229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a4807db50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c000c00 tx=0x7f6a3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a2ffff700 1 -- 172.21.15.31:0/3511775229 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a3c003db0 con 0x7f6a48108600 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a2ffff700 1 -- 172.21.15.31:0/3511775229 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c011630 con 0x7f6a48108600 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/3511775229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4807c280 con 0x7f6a48108600 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a2ffff700 1 -- 172.21.15.31:0/3511775229 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a3c011950 con 0x7f6a48108600 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.954+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/3511775229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4807c7c0 con 0x7f6a48108600 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.955+0000 7f6a2ffff700 1 -- 172.21.15.31:0/3511775229 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6a3c011af0 con 0x7f6a48108600 2024-04-25T13:09:35.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.955+0000 7f6a2ffff700 1 -- 172.21.15.31:0/3511775229 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6a3c01c450 con 0x7f6a48108600 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.955+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/3511775229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a480640a0 con 0x7f6a48108600 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:34.958+0000 7f6a2ffff700 1 -- 172.21.15.31:0/3511775229 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6a3c032020 con 0x7f6a48108600 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:35.031+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/3511775229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f6a48195370 con 0x7f6a48108600 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:35.031+0000 7f6a2ffff700 1 -- 172.21.15.31:0/3511775229 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f6a3c004260 con 0x7f6a48108600 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:35.032+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/3511775229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 msgr2=0x7f6a4807db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:35.032+0000 7f6a4d1e8700 1 --2- 172.21.15.31:0/3511775229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a4807db50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c000c00 tx=0x7f6a3c004750 comp rx=0 tx=0).stop 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:35.032+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/3511775229 shutdown_connections 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:35.032+0000 7f6a4d1e8700 1 --2- 172.21.15.31:0/3511775229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a48108600 0x7f6a4807db50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:35.032+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/3511775229 >> 172.21.15.31:0/3511775229 conn(0x7f6a48103bc0 msgr2=0x7f6a48192860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:35.032+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/3511775229 shutdown_connections 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:35.032+0000 7f6a4d1e8700 1 -- 172.21.15.31:0/3511775229 wait complete. 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2024-04-25T13:09:35.693 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: monmap epoch 1 2024-04-25T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: last_changed 2024-04-25T13:09:31.556580+0000 2024-04-25T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: created 2024-04-25T13:09:31.556580+0000 2024-04-25T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: min_mon_release 19 (squid) 2024-04-25T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: election_strategy: 1 2024-04-25T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: fsmap 2024-04-25T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: mgrmap e1: no daemons active 2024-04-25T13:09:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:35 smithi031 ceph-mon[29083]: from='client.? 172.21.15.31:0/3511775229' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T13:09:36.803 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:36 smithi031 ceph-mon[29083]: from='client.? 172.21.15.31:0/1362499219' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T13:09:36.803 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:36 smithi031 ceph-mon[29083]: from='client.? 172.21.15.31:0/1362499219' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T13:09:36.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T13:09:36.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T13:09:36.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:09:36.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T13:09:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.154+0000 7f6cf48f5700 1 Processor -- start 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.154+0000 7f6cf48f5700 1 -- start start 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.155+0000 7f6cf48f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf010b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.155+0000 7f6cf48f5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf007be50 con 0x7f6cf01090a0 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.155+0000 7f6cee59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf010b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.155+0000 7f6cee59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf010b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45780/0 (socket says 172.21.15.31:45780) 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.155+0000 7f6cee59c700 1 -- 172.21.15.31:0/1166534044 learned_addr learned my addr 172.21.15.31:0/1166534044 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.155+0000 7f6cee59c700 1 -- 172.21.15.31:0/1166534044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf010b9d0 con 0x7f6cf01090a0 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.155+0000 7f6cee59c700 1 --2- 172.21.15.31:0/1166534044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf010b490 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40098a0 tx=0x7f6ce4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155a9fa88be5b7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.155+0000 7f6ced59a700 1 -- 172.21.15.31:0/1166534044 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce4004160 con 0x7f6cf01090a0 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.156+0000 7f6ced59a700 1 -- 172.21.15.31:0/1166534044 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce4004300 con 0x7f6cf01090a0 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.156+0000 7f6ced59a700 1 -- 172.21.15.31:0/1166534044 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce40036a0 con 0x7f6cf01090a0 2024-04-25T13:09:36.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.156+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1166534044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 msgr2=0x7f6cf010b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.156+0000 7f6cf48f5700 1 --2- 172.21.15.31:0/1166534044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf010b490 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40098a0 tx=0x7f6ce4009bb0 comp rx=0 tx=0).stop 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.156+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1166534044 shutdown_connections 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.156+0000 7f6cf48f5700 1 --2- 172.21.15.31:0/1166534044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf010b490 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.156+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1166534044 >> 172.21.15.31:0/1166534044 conn(0x7f6cf0102a90 msgr2=0x7f6cf0104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.156+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1166534044 shutdown_connections 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.156+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1166534044 wait complete. 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.157+0000 7f6cf48f5700 1 Processor -- start 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.157+0000 7f6cf48f5700 1 -- start start 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.157+0000 7f6cf48f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.157+0000 7f6cf48f5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf010be40 con 0x7f6cf01090a0 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.157+0000 7f6cee59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.157+0000 7f6cee59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf019bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45794/0 (socket says 172.21.15.31:45794) 2024-04-25T13:09:36.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.157+0000 7f6cee59c700 1 -- 172.21.15.31:0/1362499219 learned_addr learned my addr 172.21.15.31:0/1362499219 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.158+0000 7f6cee59c700 1 -- 172.21.15.31:0/1362499219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf019c170 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.158+0000 7f6cee59c700 1 --2- 172.21.15.31:0/1362499219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf019bc30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4000c00 tx=0x7f6ce4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.158+0000 7f6cdf7fe700 1 -- 172.21.15.31:0/1362499219 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce4003db0 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.158+0000 7f6cdf7fe700 1 -- 172.21.15.31:0/1362499219 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce4011630 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.158+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1362499219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf019c400 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.158+0000 7f6cdf7fe700 1 -- 172.21.15.31:0/1362499219 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce4011950 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.158+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1362499219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf019f0f0 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.158+0000 7f6cdf7fe700 1 -- 172.21.15.31:0/1362499219 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6ce4011af0 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.158+0000 7f6cdf7fe700 1 -- 172.21.15.31:0/1362499219 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ce401c450 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.159+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1362499219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf00640a0 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.161+0000 7f6cdf7fe700 1 -- 172.21.15.31:0/1362499219 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6ce4004260 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.231+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1362499219 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f6cf002e110 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.234+0000 7f6cdf7fe700 1 -- 172.21.15.31:0/1362499219 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f6ce401b450 con 0x7f6cf01090a0 2024-04-25T13:09:36.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.235+0000 7f6cdf7fe700 1 -- 172.21.15.31:0/1362499219 <== mon.0 v2:172.21.15.31:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ce401b630 con 0x7f6cf01090a0 2024-04-25T13:09:36.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.236+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1362499219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 msgr2=0x7f6cf019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:36.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.236+0000 7f6cf48f5700 1 --2- 172.21.15.31:0/1362499219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf019bc30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4000c00 tx=0x7f6ce4004750 comp rx=0 tx=0).stop 2024-04-25T13:09:36.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.236+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1362499219 shutdown_connections 2024-04-25T13:09:36.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.236+0000 7f6cf48f5700 1 --2- 172.21.15.31:0/1362499219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cf01090a0 0x7f6cf019bc30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:36.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.236+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1362499219 >> 172.21.15.31:0/1362499219 conn(0x7f6cf0102a90 msgr2=0x7f6cf0103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:36.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.236+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1362499219 shutdown_connections 2024-04-25T13:09:36.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:36.236+0000 7f6cf48f5700 1 -- 172.21.15.31:0/1362499219 wait complete. 2024-04-25T13:09:36.816 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2024-04-25T13:09:37.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:37 smithi031 ceph-mon[29083]: from='client.? 172.21.15.31:0/1891226724' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:09:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.252+0000 7f8b9efa0700 1 Processor -- start 2024-04-25T13:09:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.253+0000 7f8b9efa0700 1 -- start start 2024-04-25T13:09:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.253+0000 7f8b9efa0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b98108850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.253+0000 7f8b9efa0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98108e20 con 0x7f8b98108430 2024-04-25T13:09:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.253+0000 7f8b9df9e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b98108850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.253+0000 7f8b9df9e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b98108850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45806/0 (socket says 172.21.15.31:45806) 2024-04-25T13:09:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.253+0000 7f8b9df9e700 1 -- 172.21.15.31:0/1296840474 learned_addr learned my addr 172.21.15.31:0/1296840474 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.253+0000 7f8b9df9e700 1 -- 172.21.15.31:0/1296840474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b981095e0 con 0x7f8b98108430 2024-04-25T13:09:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.254+0000 7f8b9df9e700 1 --2- 172.21.15.31:0/1296840474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b98108850 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8b940098a0 tx=0x7f8b94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dc9925f62671d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:38.041 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.254+0000 7f8b9cf9c700 1 -- 172.21.15.31:0/1296840474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b94004160 con 0x7f8b98108430 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.254+0000 7f8b9cf9c700 1 -- 172.21.15.31:0/1296840474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8b94004300 con 0x7f8b98108430 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.254+0000 7f8b9cf9c700 1 -- 172.21.15.31:0/1296840474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b94011af0 con 0x7f8b98108430 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.254+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1296840474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 msgr2=0x7f8b98108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.254+0000 7f8b9efa0700 1 --2- 172.21.15.31:0/1296840474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b98108850 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8b940098a0 tx=0x7f8b94009bb0 comp rx=0 tx=0).stop 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.254+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1296840474 shutdown_connections 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.254+0000 7f8b9efa0700 1 --2- 172.21.15.31:0/1296840474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b98108850 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.254+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1296840474 >> 172.21.15.31:0/1296840474 conn(0x7f8b98103a50 msgr2=0x7f8b98105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.255+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1296840474 shutdown_connections 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.255+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1296840474 wait complete. 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.255+0000 7f8b9efa0700 1 Processor -- start 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.255+0000 7f8b9efa0700 1 -- start start 2024-04-25T13:09:38.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.255+0000 7f8b9efa0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b9807db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.255+0000 7f8b9efa0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9810c810 con 0x7f8b98108430 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.255+0000 7f8b9df9e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b9807db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.255+0000 7f8b9df9e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b9807db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45822/0 (socket says 172.21.15.31:45822) 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.255+0000 7f8b9df9e700 1 -- 172.21.15.31:0/1891226724 learned_addr learned my addr 172.21.15.31:0/1891226724 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.256+0000 7f8b9df9e700 1 -- 172.21.15.31:0/1891226724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9807e090 con 0x7f8b98108430 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.256+0000 7f8b9df9e700 1 --2- 172.21.15.31:0/1891226724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b9807db50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009fb0 tx=0x7f8b94011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.256+0000 7f8b8affd700 1 -- 172.21.15.31:0/1891226724 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b940036a0 con 0x7f8b98108430 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.256+0000 7f8b8affd700 1 -- 172.21.15.31:0/1891226724 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8b9401a430 con 0x7f8b98108430 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.256+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1891226724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9807c280 con 0x7f8b98108430 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.256+0000 7f8b8affd700 1 -- 172.21.15.31:0/1891226724 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b94019550 con 0x7f8b98108430 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.256+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1891226724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b9807c7c0 con 0x7f8b98108430 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.256+0000 7f8b8affd700 1 -- 172.21.15.31:0/1891226724 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8b94019b90 con 0x7f8b98108430 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.256+0000 7f8b8affd700 1 -- 172.21.15.31:0/1891226724 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8b94024730 con 0x7f8b98108430 2024-04-25T13:09:38.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.257+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1891226724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b9810c990 con 0x7f8b98108430 2024-04-25T13:09:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.259+0000 7f8b8affd700 1 -- 172.21.15.31:0/1891226724 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8b94043e50 con 0x7f8b98108430 2024-04-25T13:09:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.330+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1891226724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f8b9807ceb0 con 0x7f8b98108430 2024-04-25T13:09:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.330+0000 7f8b8affd700 1 -- 172.21.15.31:0/1891226724 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f8b94032030 con 0x7f8b98108430 2024-04-25T13:09:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.331+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1891226724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 msgr2=0x7f8b9807db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.331+0000 7f8b9efa0700 1 --2- 172.21.15.31:0/1891226724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b9807db50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009fb0 tx=0x7f8b94011940 comp rx=0 tx=0).stop 2024-04-25T13:09:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.331+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1891226724 shutdown_connections 2024-04-25T13:09:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.331+0000 7f8b9efa0700 1 --2- 172.21.15.31:0/1891226724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b98108430 0x7f8b9807db50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.331+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1891226724 >> 172.21.15.31:0/1891226724 conn(0x7f8b98103a50 msgr2=0x7f8b98104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.331+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1891226724 shutdown_connections 2024-04-25T13:09:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:37.331+0000 7f8b9efa0700 1 -- 172.21.15.31:0/1891226724 wait complete. 2024-04-25T13:09:38.044 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2024-04-25T13:09:38.287 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:38 smithi031 systemd[1]: Stopping Ceph mon.a for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:09:38.596 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:38 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a[29079]: 2024-04-25T13:09:38.287+0000 7f7d23270700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:09:38.596 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:38 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a[29079]: 2024-04-25T13:09:38.287+0000 7f7d23270700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T13:09:38.596 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:38 smithi031 podman[29326]: 2024-04-25 13:09:38.369894049 +0000 UTC m=+0.156398169 container died ab6114c1a95941cfcefe509435d66a8860368f88a13055c670f7c0c3949bed37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:09:38.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:38 smithi031 podman[29326]: 2024-04-25 13:09:38.679164733 +0000 UTC m=+0.465668867 container cleanup ab6114c1a95941cfcefe509435d66a8860368f88a13055c670f7c0c3949bed37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T13:09:38.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:38 smithi031 bash[29326]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a 2024-04-25T13:09:39.373 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 podman[29340]: 2024-04-25 13:09:39.013443206 +0000 UTC m=+0.632484643 container remove ab6114c1a95941cfcefe509435d66a8860368f88a13055c670f7c0c3949bed37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T13:09:39.373 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 systemd[1]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T13:09:39.373 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 systemd[1]: Stopped Ceph mon.a for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:09:39.374 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 systemd[1]: Starting Ceph mon.a for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:09:39.634 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 podman[29438]: 2024-04-25 13:09:39.277121189 +0000 UTC m=+0.018159710 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:09:39.634 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 podman[29438]: 2024-04-25 13:09:39.43094189 +0000 UTC m=+0.171980397 container create ab96c980d7ccb9e039321d7aeee40da8501cbb9cac416864020873e24024f74f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T13:09:39.634 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 podman[29438]: 2024-04-25 13:09:39.608335683 +0000 UTC m=+0.349374201 container init ab96c980d7ccb9e039321d7aeee40da8501cbb9cac416864020873e24024f74f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:09:39.634 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 podman[29438]: 2024-04-25 13:09:39.611624508 +0000 UTC m=+0.352663017 container start ab96c980d7ccb9e039321d7aeee40da8501cbb9cac416864020873e24024f74f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T13:09:39.711 INFO:teuthology.orchestra.run.smithi031.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T13:09:39.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T13:09:39.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T13:09:39.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: pidfile_write: ignore empty --pid-file 2024-04-25T13:09:39.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: load: jerasure load: lrc 2024-04-25T13:09:39.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: RocksDB version: 7.9.2 2024-04-25T13:09:39.936 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Git sha 0 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: DB SUMMARY 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: DB Session ID: L2M8R3D68ZVSGTLGLELY 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: CURRENT file: CURRENT 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104884 ; 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.error_if_exists: 0 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.create_if_missing: 0 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.paranoid_checks: 1 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T13:09:39.937 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.env: 0x562b5d8d6cc0 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.info_log: 0x562b5fafbbe0 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.statistics: (nil) 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.use_fsync: 0 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_log_file_size: 0 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T13:09:39.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.allow_fallocate: 1 2024-04-25T13:09:39.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T13:09:39.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T13:09:39.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.use_direct_reads: 0 2024-04-25T13:09:39.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T13:09:39.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T13:09:39.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.db_log_dir: 2024-04-25T13:09:39.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.wal_dir: 2024-04-25T13:09:39.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T13:09:39.940 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.write_buffer_manager: 0x562b5fa8f0e0 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T13:09:39.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.unordered_write: 0 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.row_cache: None 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.wal_filter: None 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.two_write_queues: 0 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.wal_compression: 0 2024-04-25T13:09:39.942 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.atomic_flush: 0 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.log_readahead_size: 0 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_background_jobs: 2 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_background_compactions: -1 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_subcompactions: 1 2024-04-25T13:09:39.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_open_files: -1 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_background_flushes: -1 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Compression algorithms supported: 2024-04-25T13:09:39.944 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: kZSTD supported: 0 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: kXpressCompression supported: 0 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: kZlibCompression supported: 1 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: kSnappyCompression supported: 1 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: kLZ4Compression supported: 1 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: kBZip2Compression supported: 0 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.merge_operator: 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_filter: None 2024-04-25T13:09:39.945 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562b5fafbdc0) 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_type: 0 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_index_type: 0 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_shortening: 1 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout: checksum: 4 2024-04-25T13:09:39.946 INFO:journalctl@ceph.mon.a.smithi031.stdout: no_block_cache: 0 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache: 0x562b5fac11f0 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_options: 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: capacity : 536870912 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_shard_bits : 4 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: strict_capacity_limit : 0 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_compressed: (nil) 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: persistent_cache: (nil) 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size: 4096 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size_deviation: 10 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_restart_interval: 16 2024-04-25T13:09:39.947 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_block_restart_interval: 1 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: metadata_block_size: 4096 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: partition_filters: 0 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: use_delta_encoding: 1 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: filter_policy: bloomfilter 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: whole_key_filtering: 1 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: verify_compression: 0 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: format_version: 5 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_index_compression: 1 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_align: 0 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: max_auto_readahead_size: 262144 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: prepopulate_block_cache: 0 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: initial_auto_readahead_size: 8192 2024-04-25T13:09:39.948 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compression: NoCompression 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.num_levels: 7 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T13:09:39.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T13:09:39.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T13:09:39.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T13:09:39.952 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.inplace_update_support: 0 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.bloom_locality: 0 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.max_successive_merges: 0 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T13:09:39.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.ttl: 2592000 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.enable_blob_files: false 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.min_blob_size: 0 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T13:09:39.954 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8165e05b-8766-4890-95f9-c0077c8dfcbe 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050579641773, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050579643337, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97914, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14131, "raw_average_key_size": 49, "raw_value_size": 90258, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714050579, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8165e05b-8766-4890-95f9-c0077c8dfcbe", "db_session_id": "L2M8R3D68ZVSGTLGLELY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050579643451, "job": 1, "event": "recovery_finished"} 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562b5fbd6000 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: DB pointer 0x562b5fbc0000 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** DB Stats ** 2024-04-25T13:09:39.955 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: L0 2/0 99.55 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 106.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: Sum 2/0 99.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 106.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 106.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:09:39.956 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 106.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative compaction: 0.00 GB write, 20.78 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:09:39.957 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval compaction: 0.00 GB write, 20.78 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache BinnedLRUCache@0x562b5fac11f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: starting mon.a rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: mon.a@-1(???) e1 preinit fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: mon.a@-1(???).mds e1 new map 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: mon.a@-1(???).mds e1 print_map 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout: e1 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:09:39.958 INFO:journalctl@ceph.mon.a.smithi031.stdout: legacy client fscid: -1 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout: No filesystems configured 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: monmap epoch 1 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 bash[29438]: ab96c980d7ccb9e039321d7aeee40da8501cbb9cac416864020873e24024f74f 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: last_changed 2024-04-25T13:09:31.556580+0000 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: created 2024-04-25T13:09:31.556580+0000 2024-04-25T13:09:39.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: min_mon_release 19 (squid) 2024-04-25T13:09:39.960 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: election_strategy: 1 2024-04-25T13:09:39.960 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T13:09:39.960 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: fsmap 2024-04-25T13:09:39.960 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T13:09:39.960 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 ceph-mon[29457]: mgrmap e1: no daemons active 2024-04-25T13:09:39.960 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:39 smithi031 systemd[1]: Started Ceph mon.a for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:09:40.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.132+0000 7fab3c1e2700 1 Processor -- start 2024-04-25T13:09:40.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.132+0000 7fab3c1e2700 1 -- start start 2024-04-25T13:09:40.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.132+0000 7fab3c1e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:40.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.132+0000 7fab3c1e2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fab3407c800 con 0x7fab3407de10 2024-04-25T13:09:40.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.133+0000 7fab39f7e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:40.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.133+0000 7fab39f7e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45826/0 (socket says 172.21.15.31:45826) 2024-04-25T13:09:40.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.133+0000 7fab39f7e700 1 -- 172.21.15.31:0/2568190484 learned_addr learned my addr 172.21.15.31:0/2568190484 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:40.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.133+0000 7fab39f7e700 1 -- 172.21.15.31:0/2568190484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3407c980 con 0x7fab3407de10 2024-04-25T13:09:40.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.133+0000 7fab39f7e700 1 --2- 172.21.15.31:0/2568190484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3407c2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009e80 tx=0x7fab2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6790e7c91ad4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:40.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.133+0000 7fab38f7c700 1 -- 172.21.15.31:0/2568190484 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab2c017070 con 0x7fab3407de10 2024-04-25T13:09:40.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.133+0000 7fab38f7c700 1 -- 172.21.15.31:0/2568190484 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fab2c00b8c0 con 0x7fab3407de10 2024-04-25T13:09:40.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.133+0000 7fab38f7c700 1 -- 172.21.15.31:0/2568190484 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab2c004490 con 0x7fab3407de10 2024-04-25T13:09:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.134+0000 7fab3c1e2700 1 -- 172.21.15.31:0/2568190484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 msgr2=0x7fab3407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.134+0000 7fab3c1e2700 1 --2- 172.21.15.31:0/2568190484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3407c2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009e80 tx=0x7fab2c00b330 comp rx=0 tx=0).stop 2024-04-25T13:09:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.134+0000 7fab3c1e2700 1 -- 172.21.15.31:0/2568190484 shutdown_connections 2024-04-25T13:09:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.134+0000 7fab3c1e2700 1 --2- 172.21.15.31:0/2568190484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3407c2c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.134+0000 7fab3c1e2700 1 -- 172.21.15.31:0/2568190484 >> 172.21.15.31:0/2568190484 conn(0x7fab34103bc0 msgr2=0x7fab34105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.134+0000 7fab3c1e2700 1 -- 172.21.15.31:0/2568190484 shutdown_connections 2024-04-25T13:09:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.134+0000 7fab3c1e2700 1 -- 172.21.15.31:0/2568190484 wait complete. 2024-04-25T13:09:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab3c1e2700 1 Processor -- start 2024-04-25T13:09:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab3c1e2700 1 -- start start 2024-04-25T13:09:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab3c1e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab3c1e2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410eab0 con 0x7fab3407de10 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab39f7e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab39f7e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45834/0 (socket says 172.21.15.31:45834) 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab39f7e700 1 -- 172.21.15.31:0/3885613509 learned_addr learned my addr 172.21.15.31:0/3885613509 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab39f7e700 1 -- 172.21.15.31:0/3885613509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3419eb60 con 0x7fab3407de10 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab39f7e700 1 --2- 172.21.15.31:0/3885613509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3419e620 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00b7b0 tx=0x7fab2c00bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab32ffd700 1 -- 172.21.15.31:0/3885613509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab2c020020 con 0x7fab3407de10 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab32ffd700 1 -- 172.21.15.31:0/3885613509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fab2c012430 con 0x7fab3407de10 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.135+0000 7fab3c1e2700 1 -- 172.21.15.31:0/3885613509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3419edf0 con 0x7fab3407de10 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.136+0000 7fab32ffd700 1 -- 172.21.15.31:0/3885613509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab2c017040 con 0x7fab3407de10 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.136+0000 7fab3c1e2700 1 -- 172.21.15.31:0/3885613509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a1ae0 con 0x7fab3407de10 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.136+0000 7fab32ffd700 1 -- 172.21.15.31:0/3885613509 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab2c023020 con 0x7fab3407de10 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.136+0000 7fab32ffd700 1 -- 172.21.15.31:0/3885613509 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fab2c00f050 con 0x7fab3407de10 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.136+0000 7fab3c1e2700 1 -- 172.21.15.31:0/3885613509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab340640a0 con 0x7fab3407de10 2024-04-25T13:09:40.915 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.139+0000 7fab32ffd700 1 -- 172.21.15.31:0/3885613509 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fab2c02a020 con 0x7fab3407de10 2024-04-25T13:09:40.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.210+0000 7fab3c1e2700 1 -- 172.21.15.31:0/3885613509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fab341a1cb0 con 0x7fab3407de10 2024-04-25T13:09:40.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.212+0000 7fab32ffd700 1 -- 172.21.15.31:0/3885613509 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fab2c00b860 con 0x7fab3407de10 2024-04-25T13:09:40.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.213+0000 7fab3c1e2700 1 -- 172.21.15.31:0/3885613509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 msgr2=0x7fab3419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:40.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.213+0000 7fab3c1e2700 1 --2- 172.21.15.31:0/3885613509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3419e620 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00b7b0 tx=0x7fab2c00bdc0 comp rx=0 tx=0).stop 2024-04-25T13:09:40.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.213+0000 7fab3c1e2700 1 -- 172.21.15.31:0/3885613509 shutdown_connections 2024-04-25T13:09:40.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.213+0000 7fab3c1e2700 1 --2- 172.21.15.31:0/3885613509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3407de10 0x7fab3419e620 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:40.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.213+0000 7fab3c1e2700 1 -- 172.21.15.31:0/3885613509 >> 172.21.15.31:0/3885613509 conn(0x7fab34103bc0 msgr2=0x7fab34104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:40.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.213+0000 7fab3c1e2700 1 -- 172.21.15.31:0/3885613509 shutdown_connections 2024-04-25T13:09:40.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:40.213+0000 7fab3c1e2700 1 -- 172.21.15.31:0/3885613509 wait complete. 2024-04-25T13:09:40.916 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T13:09:40.937 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:09:40.937 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2024-04-25T13:09:40.938 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T13:09:40.938 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T13:09:41.198 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.x 2024-04-25T13:09:41.198 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T13:09:41.353 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e4b023f4-0304-11ef-bc93-c7b262605968.target.wants/ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-e4b023f4-0304-11ef-bc93-c7b262605968@.service. 2024-04-25T13:09:41.372 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:41 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3885613509' entity='client.admin' 2024-04-25T13:09:42.175 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:09:42.175 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T13:09:42.183 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:09:42.183 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T13:09:42.184 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T13:09:42.184 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T13:09:42.192 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:09:42.193 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T13:09:42.201 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:09:42.201 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T13:09:42.201 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T13:09:42.201 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T13:09:42.201 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2024-04-25T13:09:42.202 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2024-04-25T13:09:42.878 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:42 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:42.556+0000 7fcacc8da200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:09:42.878 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:42 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:42.879+0000 7fcacc8da200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:09:43.233 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:43 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:42.966+0000 7fcacc8da200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:09:44.670 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:44 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:44.261+0000 7fcacc8da200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:09:44.671 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:44 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:44.439+0000 7fcacc8da200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:09:44.671 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:44 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:44.615+0000 7fcacc8da200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:09:44.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:44 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:44.915+0000 7fcacc8da200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:09:45.299 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:45.011+0000 7fcacc8da200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:09:45.300 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:45.222+0000 7fcacc8da200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:09:45.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:45 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2261741153' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:09:45.733 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:45.428+0000 7fcacc8da200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:09:46.483 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:46 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:46.056+0000 7fcacc8da200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:09:46.801 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:46 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:46.494+0000 7fcacc8da200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:09:46.801 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:46 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:46.594+0000 7fcacc8da200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:09:46.801 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:46 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:46.801+0000 7fcacc8da200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:09:47.694 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:47 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:47.346+0000 7fcacc8da200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:09:47.694 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:47 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:47.442+0000 7fcacc8da200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:09:47.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:47 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:47.695+0000 7fcacc8da200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:09:47.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:47 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:47.833+0000 7fcacc8da200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:09:47.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:47 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:47.931+0000 7fcacc8da200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:09:48.366 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:48 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:48.052+0000 7fcacc8da200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:09:48.733 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:48 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:48.367+0000 7fcacc8da200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:09:49.220 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:48 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:48.996+0000 7fcacc8da200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:09:49.220 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:49.220+0000 7fcacc8da200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:09:49.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: Activating manager daemon x 2024-04-25T13:09:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: mgrmap e2: x(active, starting, since 0.0031961s) 2024-04-25T13:09:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: from='mgr.14100 172.21.15.31:0/3099411226' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:09:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: from='mgr.14100 172.21.15.31:0/3099411226' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:09:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: from='mgr.14100 172.21.15.31:0/3099411226' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:09:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: from='mgr.14100 172.21.15.31:0/3099411226' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:09:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: from='mgr.14100 172.21.15.31:0/3099411226' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:09:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: Manager daemon x is now available 2024-04-25T13:09:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: from='mgr.14100 172.21.15.31:0/3099411226' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:09:49.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: from='mgr.14100 172.21.15.31:0/3099411226' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:09:49.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: from='mgr.14100 172.21.15.31:0/3099411226' entity='mgr.x' 2024-04-25T13:09:49.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: from='mgr.14100 172.21.15.31:0/3099411226' entity='mgr.x' 2024-04-25T13:09:49.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:49 smithi031 ceph-mon[29457]: from='mgr.14100 172.21.15.31:0/3099411226' entity='mgr.x' 2024-04-25T13:09:51.392 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T13:09:51.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:09:51.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "e4b023f4-0304-11ef-bc93-c7b262605968", 2024-04-25T13:09:51.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:09:51.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:09:51.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:09:51.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:09:51.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:51.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:09:51.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:09:51.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:09:51.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:09:51.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:09:51.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:09:51.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:09:34.490965+0000", 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.299+0000 7f87caea4700 1 Processor -- start 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.300+0000 7f87caea4700 1 -- start start 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.300+0000 7f87caea4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c4107050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.300+0000 7f87caea4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f87c4107620 con 0x7f87c4106c30 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.300+0000 7f87c8c40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c4107050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.300+0000 7f87c8c40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c4107050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33546/0 (socket says 172.21.15.31:33546) 2024-04-25T13:09:51.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.300+0000 7f87c8c40700 1 -- 172.21.15.31:0/3708232171 learned_addr learned my addr 172.21.15.31:0/3708232171 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.300+0000 7f87c8c40700 1 -- 172.21.15.31:0/3708232171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c4107e30 con 0x7f87c4106c30 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.301+0000 7f87c8c40700 1 --2- 172.21.15.31:0/3708232171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c4107050 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f87b80098a0 tx=0x7f87b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d468349eb932dd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.301+0000 7f87c37fe700 1 -- 172.21.15.31:0/3708232171 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87b8004160 con 0x7f87c4106c30 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.301+0000 7f87c37fe700 1 -- 172.21.15.31:0/3708232171 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f87b8004300 con 0x7f87c4106c30 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.301+0000 7f87c37fe700 1 -- 172.21.15.31:0/3708232171 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87b8011af0 con 0x7f87c4106c30 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.302+0000 7f87caea4700 1 -- 172.21.15.31:0/3708232171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 msgr2=0x7f87c4107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.302+0000 7f87caea4700 1 --2- 172.21.15.31:0/3708232171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c4107050 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f87b80098a0 tx=0x7f87b8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.302+0000 7f87caea4700 1 -- 172.21.15.31:0/3708232171 shutdown_connections 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.302+0000 7f87caea4700 1 --2- 172.21.15.31:0/3708232171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c4107050 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.302+0000 7f87caea4700 1 -- 172.21.15.31:0/3708232171 >> 172.21.15.31:0/3708232171 conn(0x7f87c41021d0 msgr2=0x7f87c4104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.302+0000 7f87caea4700 1 -- 172.21.15.31:0/3708232171 shutdown_connections 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.302+0000 7f87caea4700 1 -- 172.21.15.31:0/3708232171 wait complete. 2024-04-25T13:09:51.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.303+0000 7f87caea4700 1 Processor -- start 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.303+0000 7f87caea4700 1 -- start start 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.303+0000 7f87caea4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c419b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.303+0000 7f87caea4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f87c410b060 con 0x7f87c4106c30 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.303+0000 7f87c8c40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c419b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.303+0000 7f87c8c40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c419b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33550/0 (socket says 172.21.15.31:33550) 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.303+0000 7f87c8c40700 1 -- 172.21.15.31:0/2261741153 learned_addr learned my addr 172.21.15.31:0/2261741153 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.303+0000 7f87c8c40700 1 -- 172.21.15.31:0/2261741153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c419ba30 con 0x7f87c4106c30 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.303+0000 7f87c8c40700 1 --2- 172.21.15.31:0/2261741153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c419b4f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f87b80117b0 tx=0x7f87b8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.304+0000 7f87c1ffb700 1 -- 172.21.15.31:0/2261741153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87b8003af0 con 0x7f87c4106c30 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.304+0000 7f87c1ffb700 1 -- 172.21.15.31:0/2261741153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f87b8020070 con 0x7f87c4106c30 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.304+0000 7f87caea4700 1 -- 172.21.15.31:0/2261741153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c419bcc0 con 0x7f87c4106c30 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.304+0000 7f87c1ffb700 1 -- 172.21.15.31:0/2261741153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87b8019630 con 0x7f87c4106c30 2024-04-25T13:09:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.304+0000 7f87caea4700 1 -- 172.21.15.31:0/2261741153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c419e9b0 con 0x7f87c4106c30 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.304+0000 7f87c1ffb700 1 -- 172.21.15.31:0/2261741153 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f87b8019c80 con 0x7f87c4106c30 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.304+0000 7f87c1ffb700 1 -- 172.21.15.31:0/2261741153 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f87b800b720 con 0x7f87c4106c30 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.305+0000 7f87caea4700 1 -- 172.21.15.31:0/2261741153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c410b1e0 con 0x7f87c4106c30 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.307+0000 7f87c1ffb700 1 -- 172.21.15.31:0/2261741153 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f87b80042a0 con 0x7f87c4106c30 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.381+0000 7f87caea4700 1 -- 172.21.15.31:0/2261741153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f87c4194a50 con 0x7f87c4106c30 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.381+0000 7f87c1ffb700 1 -- 172.21.15.31:0/2261741153 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f87b801a9c0 con 0x7f87c4106c30 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.383+0000 7f87caea4700 1 -- 172.21.15.31:0/2261741153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 msgr2=0x7f87c419b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.383+0000 7f87caea4700 1 --2- 172.21.15.31:0/2261741153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c419b4f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f87b80117b0 tx=0x7f87b8011b70 comp rx=0 tx=0).stop 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.383+0000 7f87caea4700 1 -- 172.21.15.31:0/2261741153 shutdown_connections 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.383+0000 7f87caea4700 1 --2- 172.21.15.31:0/2261741153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87c4106c30 0x7f87c419b4f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.383+0000 7f87caea4700 1 -- 172.21.15.31:0/2261741153 >> 172.21.15.31:0/2261741153 conn(0x7f87c41021d0 msgr2=0x7f87c4104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.383+0000 7f87caea4700 1 -- 172.21.15.31:0/2261741153 shutdown_connections 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:45.383+0000 7f87caea4700 1 -- 172.21.15.31:0/2261741153 wait complete. 2024-04-25T13:09:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2024-04-25T13:09:51.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:51 smithi031 ceph-mon[29457]: mgrmap e3: x(active, since 1.00725s) 2024-04-25T13:09:52.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:52 smithi031 ceph-mon[29457]: mgrmap e4: x(active, since 2s) 2024-04-25T13:09:54.369 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:54 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3777100734' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:09:54.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T13:09:54.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:09:54.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "e4b023f4-0304-11ef-bc93-c7b262605968", 2024-04-25T13:09:54.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:09:54.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:09:54.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:09:54.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:09:54.662 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:54.663 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:09:54.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:09:54.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:09:54.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:09:54.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:09:54.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:09:54.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:09:54.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:09:54.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:54.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:09:54.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:09:54.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:09:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:09:34.490965+0000", 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:09:54.674 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.789+0000 7f407ae56700 1 Processor -- start 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.789+0000 7f407ae56700 1 -- start start 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.789+0000 7f407ae56700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.789+0000 7f407ae56700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f407407c8d0 con 0x7f407407de50 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.790+0000 7f4078bf2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.790+0000 7f4078bf2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45000/0 (socket says 172.21.15.31:45000) 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.790+0000 7f4078bf2700 1 -- 172.21.15.31:0/4092828834 learned_addr learned my addr 172.21.15.31:0/4092828834 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.790+0000 7f4078bf2700 1 -- 172.21.15.31:0/4092828834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407407ca50 con 0x7f407407de50 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.790+0000 7f4078bf2700 1 --2- 172.21.15.31:0/4092828834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407407c300 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40680098a0 tx=0x7f4068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406555c3f82f1da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.791+0000 7f40737fe700 1 -- 172.21.15.31:0/4092828834 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4068004160 con 0x7f407407de50 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.791+0000 7f40737fe700 1 -- 172.21.15.31:0/4092828834 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4068004300 con 0x7f407407de50 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.791+0000 7f40737fe700 1 -- 172.21.15.31:0/4092828834 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4068011af0 con 0x7f407407de50 2024-04-25T13:09:54.675 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.791+0000 7f407ae56700 1 -- 172.21.15.31:0/4092828834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 msgr2=0x7f407407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.791+0000 7f407ae56700 1 --2- 172.21.15.31:0/4092828834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407407c300 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40680098a0 tx=0x7f4068009bb0 comp rx=0 tx=0).stop 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.791+0000 7f407ae56700 1 -- 172.21.15.31:0/4092828834 shutdown_connections 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.791+0000 7f407ae56700 1 --2- 172.21.15.31:0/4092828834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407407c300 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.791+0000 7f407ae56700 1 -- 172.21.15.31:0/4092828834 >> 172.21.15.31:0/4092828834 conn(0x7f4074103c30 msgr2=0x7f4074106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.791+0000 7f407ae56700 1 -- 172.21.15.31:0/4092828834 shutdown_connections 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.791+0000 7f407ae56700 1 -- 172.21.15.31:0/4092828834 wait complete. 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.792+0000 7f407ae56700 1 Processor -- start 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.792+0000 7f407ae56700 1 -- start start 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.792+0000 7f407ae56700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.792+0000 7f407ae56700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f407410eac0 con 0x7f407407de50 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.792+0000 7f4078bf2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.792+0000 7f4078bf2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407419e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45012/0 (socket says 172.21.15.31:45012) 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.792+0000 7f4078bf2700 1 -- 172.21.15.31:0/3777100734 learned_addr learned my addr 172.21.15.31:0/3777100734 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:54.676 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.793+0000 7f4078bf2700 1 -- 172.21.15.31:0/3777100734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407419ec00 con 0x7f407407de50 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.793+0000 7f4078bf2700 1 --2- 172.21.15.31:0/3777100734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407419e6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4068011ac0 tx=0x7f4068011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.793+0000 7f4071ffb700 1 -- 172.21.15.31:0/3777100734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4068003970 con 0x7f407407de50 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.793+0000 7f4071ffb700 1 -- 172.21.15.31:0/3777100734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4068003b10 con 0x7f407407de50 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.793+0000 7f407ae56700 1 -- 172.21.15.31:0/3777100734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407419ee90 con 0x7f407407de50 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.793+0000 7f4071ffb700 1 -- 172.21.15.31:0/3777100734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4068024610 con 0x7f407407de50 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.793+0000 7f407ae56700 1 -- 172.21.15.31:0/3777100734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40741a1b80 con 0x7f407407de50 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.794+0000 7f4071ffb700 1 -- 172.21.15.31:0/3777100734 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f406801a9e0 con 0x7f407407de50 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.794+0000 7f407ae56700 1 -- 172.21.15.31:0/3777100734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40740640a0 con 0x7f407407de50 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.794+0000 7f4071ffb700 1 --2- 172.21.15.31:0/3777100734 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f4064039560 0x7f406403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.794+0000 7f4071ffb700 1 -- 172.21.15.31:0/3777100734 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f406804d890 con 0x7f407407de50 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.794+0000 7f4073fff700 1 --2- 172.21.15.31:0/3777100734 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f4064039560 0x7f406403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.795+0000 7f4073fff700 1 --2- 172.21.15.31:0/3777100734 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f4064039560 0x7f406403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4060006fd0 tx=0x7f4060006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:53.800+0000 7f4071ffb700 1 -- 172.21.15.31:0/3777100734 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f406800b3e0 con 0x7f407407de50 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.093+0000 7f407ae56700 1 -- 172.21.15.31:0/3777100734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f40741a1d50 con 0x7f407407de50 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.094+0000 7f4071ffb700 1 -- 172.21.15.31:0/3777100734 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f406804d2b0 con 0x7f407407de50 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.097+0000 7f407ae56700 1 -- 172.21.15.31:0/3777100734 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f4064039560 msgr2=0x7f406403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.097+0000 7f407ae56700 1 --2- 172.21.15.31:0/3777100734 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f4064039560 0x7f406403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4060006fd0 tx=0x7f4060006e40 comp rx=0 tx=0).stop 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.097+0000 7f407ae56700 1 -- 172.21.15.31:0/3777100734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 msgr2=0x7f407419e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.097+0000 7f407ae56700 1 --2- 172.21.15.31:0/3777100734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407419e6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4068011ac0 tx=0x7f4068011ba0 comp rx=0 tx=0).stop 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.097+0000 7f407ae56700 1 -- 172.21.15.31:0/3777100734 shutdown_connections 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.097+0000 7f407ae56700 1 --2- 172.21.15.31:0/3777100734 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f4064039560 0x7f406403ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.097+0000 7f407ae56700 1 --2- 172.21.15.31:0/3777100734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f407407de50 0x7f407419e6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.097+0000 7f407ae56700 1 -- 172.21.15.31:0/3777100734 >> 172.21.15.31:0/3777100734 conn(0x7f4074103c30 msgr2=0x7f40741047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.097+0000 7f407ae56700 1 -- 172.21.15.31:0/3777100734 shutdown_connections 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:54.097+0000 7f407ae56700 1 -- 172.21.15.31:0/3777100734 wait complete. 2024-04-25T13:09:54.678 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2024-04-25T13:09:55.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:55 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/744518738' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T13:09:55.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T13:09:55.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T13:09:55.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:09:55.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T13:09:55.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T13:09:55.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T13:09:55.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T13:09:55.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T13:09:55.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T13:09:55.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T13:09:55.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T13:09:55.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-25T13:09:55.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.095+0000 7f78ac199700 1 Processor -- start 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.096+0000 7f78ac199700 1 -- start start 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.096+0000 7f78ac199700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a410b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.096+0000 7f78ac199700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4063d60 con 0x7f78a4109040 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.096+0000 7f78a9f35700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a410b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.096+0000 7f78a9f35700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a410b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45028/0 (socket says 172.21.15.31:45028) 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.096+0000 7f78a9f35700 1 -- 172.21.15.31:0/4266973444 learned_addr learned my addr 172.21.15.31:0/4266973444 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:55.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.096+0000 7f78a9f35700 1 -- 172.21.15.31:0/4266973444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a410b970 con 0x7f78a4109040 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.096+0000 7f78a9f35700 1 --2- 172.21.15.31:0/4266973444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a410b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f78a00098a0 tx=0x7f78a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3850ae99935bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.097+0000 7f78a8f33700 1 -- 172.21.15.31:0/4266973444 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78a0004160 con 0x7f78a4109040 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.097+0000 7f78a8f33700 1 -- 172.21.15.31:0/4266973444 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f78a0004300 con 0x7f78a4109040 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.097+0000 7f78a8f33700 1 -- 172.21.15.31:0/4266973444 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78a0011af0 con 0x7f78a4109040 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.097+0000 7f78ac199700 1 -- 172.21.15.31:0/4266973444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 msgr2=0x7f78a410b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.097+0000 7f78ac199700 1 --2- 172.21.15.31:0/4266973444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a410b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f78a00098a0 tx=0x7f78a0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.097+0000 7f78ac199700 1 -- 172.21.15.31:0/4266973444 shutdown_connections 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.097+0000 7f78ac199700 1 --2- 172.21.15.31:0/4266973444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a410b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.097+0000 7f78ac199700 1 -- 172.21.15.31:0/4266973444 >> 172.21.15.31:0/4266973444 conn(0x7f78a4102ab0 msgr2=0x7f78a4104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.097+0000 7f78ac199700 1 -- 172.21.15.31:0/4266973444 shutdown_connections 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.097+0000 7f78ac199700 1 -- 172.21.15.31:0/4266973444 wait complete. 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.098+0000 7f78ac199700 1 Processor -- start 2024-04-25T13:09:55.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.098+0000 7f78ac199700 1 -- start start 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.098+0000 7f78ac199700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a41088c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.098+0000 7f78ac199700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f78a410c8d0 con 0x7f78a4109040 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.098+0000 7f78a9f35700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a41088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.098+0000 7f78a9f35700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a41088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45038/0 (socket says 172.21.15.31:45038) 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.098+0000 7f78a9f35700 1 -- 172.21.15.31:0/744518738 learned_addr learned my addr 172.21.15.31:0/744518738 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.099+0000 7f78a9f35700 1 -- 172.21.15.31:0/744518738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a4108e00 con 0x7f78a4109040 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.099+0000 7f78a9f35700 1 --2- 172.21.15.31:0/744518738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a41088c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f78a0003f40 tx=0x7f78a0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.099+0000 7f789affd700 1 -- 172.21.15.31:0/744518738 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78a00038b0 con 0x7f78a4109040 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.099+0000 7f789affd700 1 -- 172.21.15.31:0/744518738 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f78a0003a50 con 0x7f78a4109040 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.099+0000 7f789affd700 1 -- 172.21.15.31:0/744518738 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78a001a5c0 con 0x7f78a4109040 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.099+0000 7f78ac199700 1 -- 172.21.15.31:0/744518738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a4106ff0 con 0x7f78a4109040 2024-04-25T13:09:55.926 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.099+0000 7f78ac199700 1 -- 172.21.15.31:0/744518738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a4107530 con 0x7f78a4109040 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.100+0000 7f789affd700 1 -- 172.21.15.31:0/744518738 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f78a0024430 con 0x7f78a4109040 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.100+0000 7f78ac199700 1 -- 172.21.15.31:0/744518738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a40640a0 con 0x7f78a4109040 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.100+0000 7f789affd700 1 --2- 172.21.15.31:0/744518738 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f7890039510 0x7f789003b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.100+0000 7f789affd700 1 -- 172.21.15.31:0/744518738 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f78a004da50 con 0x7f78a4109040 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.100+0000 7f78a9734700 1 --2- 172.21.15.31:0/744518738 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f7890039510 0x7f789003b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.101+0000 7f78a9734700 1 --2- 172.21.15.31:0/744518738 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f7890039510 0x7f789003b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7894006fd0 tx=0x7f7894006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.105+0000 7f789affd700 1 -- 172.21.15.31:0/744518738 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78a004d470 con 0x7f78a4109040 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.312+0000 7f78ac199700 1 -- 172.21.15.31:0/744518738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f78a41077b0 con 0x7f78a4109040 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.313+0000 7f789affd700 1 -- 172.21.15.31:0/744518738 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f78a002d910 con 0x7f78a4109040 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.316+0000 7f78ac199700 1 -- 172.21.15.31:0/744518738 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f7890039510 msgr2=0x7f789003b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.316+0000 7f78ac199700 1 --2- 172.21.15.31:0/744518738 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f7890039510 0x7f789003b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7894006fd0 tx=0x7f7894006e40 comp rx=0 tx=0).stop 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.316+0000 7f78ac199700 1 -- 172.21.15.31:0/744518738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 msgr2=0x7f78a41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.316+0000 7f78ac199700 1 --2- 172.21.15.31:0/744518738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a41088c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f78a0003f40 tx=0x7f78a0011890 comp rx=0 tx=0).stop 2024-04-25T13:09:55.927 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.316+0000 7f78ac199700 1 -- 172.21.15.31:0/744518738 shutdown_connections 2024-04-25T13:09:55.928 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.316+0000 7f78ac199700 1 --2- 172.21.15.31:0/744518738 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f7890039510 0x7f789003b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:55.928 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.316+0000 7f78ac199700 1 --2- 172.21.15.31:0/744518738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78a4109040 0x7f78a41088c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:55.928 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.316+0000 7f78ac199700 1 -- 172.21.15.31:0/744518738 >> 172.21.15.31:0/744518738 conn(0x7f78a4102ab0 msgr2=0x7f78a4103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:55.928 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.316+0000 7f78ac199700 1 -- 172.21.15.31:0/744518738 shutdown_connections 2024-04-25T13:09:55.928 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:55.316+0000 7f78ac199700 1 -- 172.21.15.31:0/744518738 wait complete. 2024-04-25T13:09:55.928 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2024-04-25T13:09:56.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:56 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3156841544' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T13:09:56.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:56 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: ignoring --setuser ceph since I am not root 2024-04-25T13:09:56.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:56 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: ignoring --setgroup ceph since I am not root 2024-04-25T13:09:56.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:56 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:56.842+0000 7fa0998c5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:09:57.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:57 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:57.038+0000 7fa0998c5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:09:57.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:57 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:57.144+0000 7fa0998c5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:09:57.465 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:57 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:57.464+0000 7fa0998c5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:09:57.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:57 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3156841544' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T13:09:57.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:57 smithi031 ceph-mon[29457]: mgrmap e5: x(active, since 7s) 2024-04-25T13:09:57.733 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:57 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:57.552+0000 7fa0998c5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:09:58.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.336+0000 7f14f50d9700 1 Processor -- start 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.336+0000 7f14f50d9700 1 -- start start 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.336+0000 7f14f50d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f01078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.336+0000 7f14f50d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14f0107ec0 con 0x7f14f01074d0 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.336+0000 7f14eed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f01078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.336+0000 7f14eed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f01078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45052/0 (socket says 172.21.15.31:45052) 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.336+0000 7f14eed9d700 1 -- 172.21.15.31:0/2292549216 learned_addr learned my addr 172.21.15.31:0/2292549216 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.337+0000 7f14eed9d700 1 -- 172.21.15.31:0/2292549216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f01086d0 con 0x7f14f01074d0 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.337+0000 7f14eed9d700 1 --2- 172.21.15.31:0/2292549216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f01078f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f14e000bb40 tx=0x7f14e000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc3c7eb7de80e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.337+0000 7f14edd9b700 1 -- 172.21.15.31:0/2292549216 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e000dbe0 con 0x7f14f01074d0 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.337+0000 7f14edd9b700 1 -- 172.21.15.31:0/2292549216 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f14e00044c0 con 0x7f14f01074d0 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.337+0000 7f14edd9b700 1 -- 172.21.15.31:0/2292549216 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e0010470 con 0x7f14f01074d0 2024-04-25T13:09:58.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.338+0000 7f14f50d9700 1 -- 172.21.15.31:0/2292549216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 msgr2=0x7f14f01078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.338+0000 7f14f50d9700 1 --2- 172.21.15.31:0/2292549216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f01078f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f14e000bb40 tx=0x7f14e000be50 comp rx=0 tx=0).stop 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.338+0000 7f14f50d9700 1 -- 172.21.15.31:0/2292549216 shutdown_connections 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.338+0000 7f14f50d9700 1 --2- 172.21.15.31:0/2292549216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f01078f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.338+0000 7f14f50d9700 1 -- 172.21.15.31:0/2292549216 >> 172.21.15.31:0/2292549216 conn(0x7f14f0102a70 msgr2=0x7f14f0104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.338+0000 7f14f50d9700 1 -- 172.21.15.31:0/2292549216 shutdown_connections 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.338+0000 7f14f50d9700 1 -- 172.21.15.31:0/2292549216 wait complete. 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.338+0000 7f14f50d9700 1 Processor -- start 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.339+0000 7f14f50d9700 1 -- start start 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.339+0000 7f14f50d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.339+0000 7f14f50d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14f007afb0 con 0x7f14f01074d0 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.339+0000 7f14eed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:58.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.339+0000 7f14eed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45068/0 (socket says 172.21.15.31:45068) 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.339+0000 7f14eed9d700 1 -- 172.21.15.31:0/3156841544 learned_addr learned my addr 172.21.15.31:0/3156841544 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.339+0000 7f14eed9d700 1 -- 172.21.15.31:0/3156841544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f019ebc0 con 0x7f14f01074d0 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.339+0000 7f14eed9d700 1 --2- 172.21.15.31:0/3156841544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f019e680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f14e000bfd0 tx=0x7f14e0003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.340+0000 7f14d7fff700 1 -- 172.21.15.31:0/3156841544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e0010770 con 0x7f14f01074d0 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.340+0000 7f14d7fff700 1 -- 172.21.15.31:0/3156841544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f14e001d070 con 0x7f14f01074d0 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.340+0000 7f14f50d9700 1 -- 172.21.15.31:0/3156841544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f019ee50 con 0x7f14f01074d0 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.340+0000 7f14d7fff700 1 -- 172.21.15.31:0/3156841544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e0019c70 con 0x7f14f01074d0 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.340+0000 7f14f50d9700 1 -- 172.21.15.31:0/3156841544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f01a1af0 con 0x7f14f01074d0 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.341+0000 7f14d7fff700 1 -- 172.21.15.31:0/3156841544 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f14e0028060 con 0x7f14f01074d0 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.341+0000 7f14f50d9700 1 -- 172.21.15.31:0/3156841544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f00640a0 con 0x7f14f01074d0 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.341+0000 7f14d7fff700 1 --2- 172.21.15.31:0/3156841544 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f14d8039510 0x7f14d803b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.341+0000 7f14d7fff700 1 -- 172.21.15.31:0/3156841544 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f14e004d960 con 0x7f14f01074d0 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.341+0000 7f14ee59c700 1 --2- 172.21.15.31:0/3156841544 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f14d8039510 0x7f14d803b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:09:58.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.341+0000 7f14ee59c700 1 --2- 172.21.15.31:0/3156841544 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f14d8039510 0x7f14d803b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f14e4006fd0 tx=0x7f14e4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.346+0000 7f14d7fff700 1 -- 172.21.15.31:0/3156841544 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14e0015070 con 0x7f14f01074d0 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.613+0000 7f14f50d9700 1 -- 172.21.15.31:0/3156841544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f14f01a1cc0 con 0x7f14f01074d0 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.664+0000 7f14d7fff700 1 -- 172.21.15.31:0/3156841544 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f14f01a1cc0 con 0x7f14f01074d0 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.666+0000 7f14d7fff700 1 -- 172.21.15.31:0/3156841544 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f14e001f030 con 0x7f14f01074d0 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.667+0000 7f14f50d9700 1 -- 172.21.15.31:0/3156841544 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f14d8039510 msgr2=0x7f14d803b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.667+0000 7f14f50d9700 1 --2- 172.21.15.31:0/3156841544 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f14d8039510 0x7f14d803b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f14e4006fd0 tx=0x7f14e4006e40 comp rx=0 tx=0).stop 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.667+0000 7f14f50d9700 1 -- 172.21.15.31:0/3156841544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 msgr2=0x7f14f019e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.667+0000 7f14f50d9700 1 --2- 172.21.15.31:0/3156841544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f019e680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f14e000bfd0 tx=0x7f14e0003ff0 comp rx=0 tx=0).stop 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.667+0000 7f14f50d9700 1 -- 172.21.15.31:0/3156841544 shutdown_connections 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.667+0000 7f14f50d9700 1 --2- 172.21.15.31:0/3156841544 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f14d8039510 0x7f14d803b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.667+0000 7f14f50d9700 1 --2- 172.21.15.31:0/3156841544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14f01074d0 0x7f14f019e680 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.667+0000 7f14f50d9700 1 -- 172.21.15.31:0/3156841544 >> 172.21.15.31:0/3156841544 conn(0x7f14f0102a70 msgr2=0x7f14f0190890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.668+0000 7f14f50d9700 1 -- 172.21.15.31:0/3156841544 shutdown_connections 2024-04-25T13:09:58.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:56.668+0000 7f14f50d9700 1 -- 172.21.15.31:0/3156841544 wait complete. 2024-04-25T13:09:59.094 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:58 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:58.803+0000 7fa0998c5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:09:59.094 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:58 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:58.980+0000 7fa0998c5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:09:59.355 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:59 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:59.164+0000 7fa0998c5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:09:59.639 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:59 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:59.473+0000 7fa0998c5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:09:59.639 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:59 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:59.570+0000 7fa0998c5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:09:59.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:09:59 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/146049270' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T13:09:59.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:59 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:59.783+0000 7fa0998c5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:10:00.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:10:00.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T13:10:00.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:10:00.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T13:10:00.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T13:10:00.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.355+0000 7ff118df5700 1 Processor -- start 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.355+0000 7ff118df5700 1 -- start start 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.355+0000 7ff118df5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff114107270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.355+0000 7ff118df5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff11407be50 con 0x7ff114106e50 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.355+0000 7ff11259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff114107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.356+0000 7ff11259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff114107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45102/0 (socket says 172.21.15.31:45102) 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.356+0000 7ff11259c700 1 -- 172.21.15.31:0/372523321 learned_addr learned my addr 172.21.15.31:0/372523321 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.356+0000 7ff11259c700 1 -- 172.21.15.31:0/372523321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1141077b0 con 0x7ff114106e50 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.356+0000 7ff11259c700 1 --2- 172.21.15.31:0/372523321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff114107270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff1040098a0 tx=0x7ff104009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f39ebde2a72308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.356+0000 7ff11159a700 1 -- 172.21.15.31:0/372523321 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff104004160 con 0x7ff114106e50 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.356+0000 7ff11159a700 1 -- 172.21.15.31:0/372523321 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff104004300 con 0x7ff114106e50 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.356+0000 7ff11159a700 1 -- 172.21.15.31:0/372523321 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff104011af0 con 0x7ff114106e50 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.357+0000 7ff118df5700 1 -- 172.21.15.31:0/372523321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 msgr2=0x7ff114107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.357+0000 7ff118df5700 1 --2- 172.21.15.31:0/372523321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff114107270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff1040098a0 tx=0x7ff104009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.357+0000 7ff118df5700 1 -- 172.21.15.31:0/372523321 shutdown_connections 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.357+0000 7ff118df5700 1 --2- 172.21.15.31:0/372523321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff114107270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.357+0000 7ff118df5700 1 -- 172.21.15.31:0/372523321 >> 172.21.15.31:0/372523321 conn(0x7ff114102a70 msgr2=0x7ff114104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.357+0000 7ff118df5700 1 -- 172.21.15.31:0/372523321 shutdown_connections 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.357+0000 7ff118df5700 1 -- 172.21.15.31:0/372523321 wait complete. 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.358+0000 7ff118df5700 1 Processor -- start 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.358+0000 7ff118df5700 1 -- start start 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.358+0000 7ff118df5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff11419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.358+0000 7ff118df5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff114108770 con 0x7ff114106e50 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.358+0000 7ff11259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff11419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.358+0000 7ff11259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff11419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45114/0 (socket says 172.21.15.31:45114) 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.358+0000 7ff11259c700 1 -- 172.21.15.31:0/146049270 learned_addr learned my addr 172.21.15.31:0/146049270 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:00.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.358+0000 7ff11259c700 1 -- 172.21.15.31:0/146049270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11419ec30 con 0x7ff114106e50 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.359+0000 7ff11259c700 1 --2- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff11419e6f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff104009fb0 tx=0x7ff104011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.359+0000 7ff1037fe700 1 -- 172.21.15.31:0/146049270 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1040036a0 con 0x7ff114106e50 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.359+0000 7ff1037fe700 1 -- 172.21.15.31:0/146049270 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff10401a430 con 0x7ff114106e50 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.359+0000 7ff118df5700 1 -- 172.21.15.31:0/146049270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11419eec0 con 0x7ff114106e50 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.359+0000 7ff1037fe700 1 -- 172.21.15.31:0/146049270 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff104019550 con 0x7ff114106e50 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.359+0000 7ff118df5700 1 -- 172.21.15.31:0/146049270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1141a1bb0 con 0x7ff114106e50 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.360+0000 7ff1037fe700 1 -- 172.21.15.31:0/146049270 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff104019800 con 0x7ff114106e50 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.360+0000 7ff1037fe700 1 --2- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7ff0fc0395b0 0x7ff0fc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.360+0000 7ff1037fe700 1 -- 172.21.15.31:0/146049270 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff10404e880 con 0x7ff114106e50 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.360+0000 7ff111d9b700 1 -- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7ff0fc0395b0 msgr2=0x7ff0fc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1458774477 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.360+0000 7ff111d9b700 1 --2- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7ff0fc0395b0 0x7ff0fc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.360+0000 7ff118df5700 1 -- 172.21.15.31:0/146049270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1140640a0 con 0x7ff114106e50 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.365+0000 7ff1037fe700 1 -- 172.21.15.31:0/146049270 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff104023c90 con 0x7ff114106e50 2024-04-25T13:10:00.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.561+0000 7ff111d9b700 1 -- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7ff0fc0395b0 msgr2=0x7ff0fc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1458774477 2024-04-25T13:10:00.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.561+0000 7ff111d9b700 1 --2- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7ff0fc0395b0 0x7ff0fc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:10:00.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.638+0000 7ff118df5700 1 -- 172.21.15.31:0/146049270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff11402e080 con 0x7ff114106e50 2024-04-25T13:10:00.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.639+0000 7ff1037fe700 1 -- 172.21.15.31:0/146049270 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7ff10404e2a0 con 0x7ff114106e50 2024-04-25T13:10:00.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.641+0000 7ff118df5700 1 -- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7ff0fc0395b0 msgr2=0x7ff0fc03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:10:00.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.641+0000 7ff118df5700 1 --2- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7ff0fc0395b0 0x7ff0fc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:00.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.641+0000 7ff118df5700 1 -- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 msgr2=0x7ff11419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:00.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.641+0000 7ff118df5700 1 --2- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff11419e6f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff104009fb0 tx=0x7ff104011940 comp rx=0 tx=0).stop 2024-04-25T13:10:00.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.641+0000 7ff118df5700 1 -- 172.21.15.31:0/146049270 shutdown_connections 2024-04-25T13:10:00.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.641+0000 7ff118df5700 1 --2- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7ff0fc0395b0 0x7ff0fc03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:00.237 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.641+0000 7ff118df5700 1 --2- 172.21.15.31:0/146049270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff114106e50 0x7ff11419e6f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:00.237 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.641+0000 7ff118df5700 1 -- 172.21.15.31:0/146049270 >> 172.21.15.31:0/146049270 conn(0x7ff114102a70 msgr2=0x7ff114195360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:00.237 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.641+0000 7ff118df5700 1 -- 172.21.15.31:0/146049270 shutdown_connections 2024-04-25T13:10:00.237 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:09:59.641+0000 7ff118df5700 1 -- 172.21.15.31:0/146049270 wait complete. 2024-04-25T13:10:00.237 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-04-25T13:10:00.237 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2024-04-25T13:10:00.349 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:09:59 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:09:59.990+0000 7fa0998c5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:10:00.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:00 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:00.636+0000 7fa0998c5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:10:01.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:01 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:01.078+0000 7fa0998c5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:10:01.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:01 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:01.177+0000 7fa0998c5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:10:01.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:01 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:01.386+0000 7fa0998c5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:10:02.233 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:01 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:01.931+0000 7fa0998c5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:10:02.233 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:02 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:02.026+0000 7fa0998c5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:10:02.636 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:02 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:02.279+0000 7fa0998c5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:10:02.637 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:02 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:02.418+0000 7fa0998c5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:10:02.637 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:02 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:02.517+0000 7fa0998c5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:10:02.637 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:02 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:02.636+0000 7fa0998c5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:10:03.233 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:02 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:02.948+0000 7fa0998c5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:10:03.852 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:03.575+0000 7fa0998c5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:10:03.852 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:03.799+0000 7fa0998c5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:10:04.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: Active manager daemon x restarted 2024-04-25T13:10:04.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: Activating manager daemon x 2024-04-25T13:10:04.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T13:10:04.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: mgrmap e6: x(active, starting, since 0.00339361s) 2024-04-25T13:10:04.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:10:04.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:10:04.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:10:04.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:10:04.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:10:04.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: Manager daemon x is now available 2024-04-25T13:10:04.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:04.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:04.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:04.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:03 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:05.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:04 smithi031 ceph-mon[29457]: Found migration_current of "None". Setting to last migration. 2024-04-25T13:10:05.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:04 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:10:05.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:04 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:10:05.065 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:04 smithi031 ceph-mon[29457]: mgrmap e7: x(active, since 1.00631s) 2024-04-25T13:10:05.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:10:05.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T13:10:05.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T13:10:05.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:10:05.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.656+0000 7f8cb1156700 1 Processor -- start 2024-04-25T13:10:05.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.656+0000 7f8cb1156700 1 -- start start 2024-04-25T13:10:05.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.656+0000 7f8cb1156700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac1089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:05.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.656+0000 7f8cb1156700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac108fa0 con 0x7f8cac1085b0 2024-04-25T13:10:05.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.656+0000 7f8caad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac1089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:05.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.657+0000 7f8caad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac1089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45124/0 (socket says 172.21.15.31:45124) 2024-04-25T13:10:05.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.657+0000 7f8caad9d700 1 -- 172.21.15.31:0/425959593 learned_addr learned my addr 172.21.15.31:0/425959593 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:05.383 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.657+0000 7f8caad9d700 1 -- 172.21.15.31:0/425959593 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac109760 con 0x7f8cac1085b0 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.657+0000 7f8caad9d700 1 --2- 172.21.15.31:0/425959593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac1089d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0098a0 tx=0x7f8c9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd07f3fdb62b571b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.657+0000 7f8ca9d9b700 1 -- 172.21.15.31:0/425959593 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c004160 con 0x7f8cac1085b0 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.657+0000 7f8ca9d9b700 1 -- 172.21.15.31:0/425959593 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8c9c004300 con 0x7f8cac1085b0 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.657+0000 7f8ca9d9b700 1 -- 172.21.15.31:0/425959593 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c011af0 con 0x7f8cac1085b0 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.658+0000 7f8cb1156700 1 -- 172.21.15.31:0/425959593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 msgr2=0x7f8cac1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.658+0000 7f8cb1156700 1 --2- 172.21.15.31:0/425959593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac1089d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0098a0 tx=0x7f8c9c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.658+0000 7f8cb1156700 1 -- 172.21.15.31:0/425959593 shutdown_connections 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.658+0000 7f8cb1156700 1 --2- 172.21.15.31:0/425959593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac1089d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.658+0000 7f8cb1156700 1 -- 172.21.15.31:0/425959593 >> 172.21.15.31:0/425959593 conn(0x7f8cac103b70 msgr2=0x7f8cac105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.658+0000 7f8cb1156700 1 -- 172.21.15.31:0/425959593 shutdown_connections 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.658+0000 7f8cb1156700 1 -- 172.21.15.31:0/425959593 wait complete. 2024-04-25T13:10:05.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.659+0000 7f8cb1156700 1 Processor -- start 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.659+0000 7f8cb1156700 1 -- start start 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.659+0000 7f8cb1156700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.659+0000 7f8cb1156700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac10c990 con 0x7f8cac1085b0 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.660+0000 7f8caad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.660+0000 7f8caad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45128/0 (socket says 172.21.15.31:45128) 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.660+0000 7f8caad9d700 1 -- 172.21.15.31:0/2002218641 learned_addr learned my addr 172.21.15.31:0/2002218641 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.660+0000 7f8caad9d700 1 -- 172.21.15.31:0/2002218641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac19ca00 con 0x7f8cac1085b0 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.660+0000 7f8caad9d700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac19c4c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c003f40 tx=0x7f8c9c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.660+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c0038b0 con 0x7f8cac1085b0 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.660+0000 7f8cb1156700 1 -- 172.21.15.31:0/2002218641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cac19cc90 con 0x7f8cac1085b0 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.660+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8c9c003a50 con 0x7f8cac1085b0 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.660+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c01a5c0 con 0x7f8cac1085b0 2024-04-25T13:10:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.660+0000 7f8cb1156700 1 -- 172.21.15.31:0/2002218641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cac19f980 con 0x7f8cac1085b0 2024-04-25T13:10:05.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.661+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f8c9c01a8c0 con 0x7f8cac1085b0 2024-04-25T13:10:05.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.661+0000 7f8c93fff700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 0x7f8c9403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:05.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.661+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 --> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8c9403c130 con 0x7f8c94039560 2024-04-25T13:10:05.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.662+0000 7f8caa59c700 1 -- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 msgr2=0x7f8c9403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1458774477 2024-04-25T13:10:05.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.662+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c9c04daf0 con 0x7f8cac1085b0 2024-04-25T13:10:05.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.662+0000 7f8caa59c700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 0x7f8c9403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:10:05.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.863+0000 7f8caa59c700 1 -- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 msgr2=0x7f8c9403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1458774477 2024-04-25T13:10:05.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:00.863+0000 7f8caa59c700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 0x7f8c9403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:10:05.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:01.264+0000 7f8caa59c700 1 -- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 msgr2=0x7f8c9403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1458774477 2024-04-25T13:10:05.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:01.264+0000 7f8caa59c700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 0x7f8c9403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:10:05.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:02.065+0000 7f8caa59c700 1 -- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 msgr2=0x7f8c9403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1458774477 2024-04-25T13:10:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:02.065+0000 7f8caa59c700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 0x7f8c9403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:10:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:03.666+0000 7f8caa59c700 1 -- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 msgr2=0x7f8c9403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1458774477 2024-04-25T13:10:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:03.666+0000 7f8caa59c700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 0x7f8c9403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T13:10:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:03.802+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f8c9c01ac00 con 0x7f8cac1085b0 2024-04-25T13:10:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:03.803+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 msgr2=0x7f8c9403ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:10:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:03.803+0000 7f8c93fff700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 0x7f8c9403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.804+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f8c9c04f9c0 con 0x7f8cac1085b0 2024-04-25T13:10:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.804+0000 7f8c93fff700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8c9403ce50 0x7f8c9403f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.804+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 --> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8c9c012ec0 con 0x7f8c9403ce50 2024-04-25T13:10:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.806+0000 7f8caa59c700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8c9403ce50 0x7f8c9403f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:05.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.806+0000 7f8caa59c700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8c9403ce50 0x7f8c9403f240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0003a10 tx=0x7f8ca0009290 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.807+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 <== mgr.14116 v2:172.21.15.31:6800/3047311576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f8c9c012ec0 con 0x7f8c9403ce50 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.815+0000 7f8cb1156700 1 -- 172.21.15.31:0/2002218641 --> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8cac19d170 con 0x7f8c9403ce50 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8c93fff700 1 -- 172.21.15.31:0/2002218641 <== mgr.14116 v2:172.21.15.31:6800/3047311576 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f8cac19d170 con 0x7f8c9403ce50 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8cb1156700 1 -- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8c9403ce50 msgr2=0x7f8c9403f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8cb1156700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8c9403ce50 0x7f8c9403f240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0003a10 tx=0x7f8ca0009290 comp rx=0 tx=0).stop 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8cb1156700 1 -- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 msgr2=0x7f8cac19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8cb1156700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac19c4c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c003f40 tx=0x7f8c9c011890 comp rx=0 tx=0).stop 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8cb1156700 1 -- 172.21.15.31:0/2002218641 shutdown_connections 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8cb1156700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8c9403ce50 0x7f8c9403f240 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8cb1156700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cac1085b0 0x7f8cac19c4c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8cb1156700 1 --2- 172.21.15.31:0/2002218641 >> [v2:172.21.15.31:6800/1458774477,v1:172.21.15.31:6801/1458774477] conn(0x7f8c94039560 0x7f8c9403ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:05.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8cb1156700 1 -- 172.21.15.31:0/2002218641 >> 172.21.15.31:0/2002218641 conn(0x7f8cac103b70 msgr2=0x7f8cac104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:05.389 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8cb1156700 1 -- 172.21.15.31:0/2002218641 shutdown_connections 2024-04-25T13:10:05.389 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:04.816+0000 7f8cb1156700 1 -- 172.21.15.31:0/2002218641 wait complete. 2024-04-25T13:10:05.389 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2024-04-25T13:10:05.389 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2024-04-25T13:10:06.393 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:06 smithi031 ceph-mon[29457]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T13:10:06.394 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:06 smithi031 ceph-mon[29457]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T13:10:06.394 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:06 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:06.394 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:06 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:06.394 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:06 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:06.394 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:06 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:06.394 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:06 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:06.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.818+0000 7f8ae5e0e700 1 Processor -- start 2024-04-25T13:10:06.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.818+0000 7f8ae5e0e700 1 -- start start 2024-04-25T13:10:06.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.819+0000 7f8ae5e0e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae01078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:06.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.819+0000 7f8ae5e0e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae0107ec0 con 0x7f8ae01074d0 2024-04-25T13:10:06.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.819+0000 7f8adf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae01078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:06.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.819+0000 7f8adf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae01078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56808/0 (socket says 172.21.15.31:56808) 2024-04-25T13:10:06.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.819+0000 7f8adf7fe700 1 -- 172.21.15.31:0/572839548 learned_addr learned my addr 172.21.15.31:0/572839548 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:06.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.819+0000 7f8adf7fe700 1 -- 172.21.15.31:0/572839548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae01086d0 con 0x7f8ae01074d0 2024-04-25T13:10:06.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.819+0000 7f8adf7fe700 1 --2- 172.21.15.31:0/572839548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae01078f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400bb40 tx=0x7f8ad400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7684b51e2b38506 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:06.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.819+0000 7f8ade7fc700 1 -- 172.21.15.31:0/572839548 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ad400dbe0 con 0x7f8ae01074d0 2024-04-25T13:10:06.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.819+0000 7f8ade7fc700 1 -- 172.21.15.31:0/572839548 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8ad40044c0 con 0x7f8ae01074d0 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.819+0000 7f8ade7fc700 1 -- 172.21.15.31:0/572839548 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ad4010470 con 0x7f8ae01074d0 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.820+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/572839548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 msgr2=0x7f8ae01078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.820+0000 7f8ae5e0e700 1 --2- 172.21.15.31:0/572839548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae01078f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400bb40 tx=0x7f8ad400be50 comp rx=0 tx=0).stop 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.820+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/572839548 shutdown_connections 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.820+0000 7f8ae5e0e700 1 --2- 172.21.15.31:0/572839548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae01078f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.820+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/572839548 >> 172.21.15.31:0/572839548 conn(0x7f8ae0102ab0 msgr2=0x7f8ae0104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.820+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/572839548 shutdown_connections 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.820+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/572839548 wait complete. 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.821+0000 7f8ae5e0e700 1 Processor -- start 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.821+0000 7f8ae5e0e700 1 -- start start 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.821+0000 7f8ae5e0e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.821+0000 7f8ae5e0e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae0062e80 con 0x7f8ae01074d0 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.821+0000 7f8adf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.821+0000 7f8adf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56818/0 (socket says 172.21.15.31:56818) 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.821+0000 7f8adf7fe700 1 -- 172.21.15.31:0/3696649489 learned_addr learned my addr 172.21.15.31:0/3696649489 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.822+0000 7f8adf7fe700 1 -- 172.21.15.31:0/3696649489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae019a790 con 0x7f8ae01074d0 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.822+0000 7f8adf7fe700 1 --2- 172.21.15.31:0/3696649489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae019a250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400bfd0 tx=0x7f8ad4003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.822+0000 7f8adcff9700 1 -- 172.21.15.31:0/3696649489 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ad4027070 con 0x7f8ae01074d0 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.822+0000 7f8adcff9700 1 -- 172.21.15.31:0/3696649489 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8ad4010c00 con 0x7f8ae01074d0 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.822+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/3696649489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae019aa20 con 0x7f8ae01074d0 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.822+0000 7f8adcff9700 1 -- 172.21.15.31:0/3696649489 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ad4022c10 con 0x7f8ae01074d0 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.822+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/3696649489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae019d710 con 0x7f8ae01074d0 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.823+0000 7f8adcff9700 1 -- 172.21.15.31:0/3696649489 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f8ad402a070 con 0x7f8ae01074d0 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.823+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/3696649489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae019af00 con 0x7f8ae01074d0 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.823+0000 7f8adcff9700 1 --2- 172.21.15.31:0/3696649489 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8ac8039450 0x7f8ac803b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.823+0000 7f8adcff9700 1 -- 172.21.15.31:0/3696649489 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8ad401f070 con 0x7f8ae01074d0 2024-04-25T13:10:06.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.823+0000 7f8adeffd700 1 --2- 172.21.15.31:0/3696649489 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8ac8039450 0x7f8ac803b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.824+0000 7f8adeffd700 1 --2- 172.21.15.31:0/3696649489 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8ac8039450 0x7f8ac803b910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0006fd0 tx=0x7f8ad0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:05.828+0000 7f8adcff9700 1 -- 172.21.15.31:0/3696649489 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ae019af00 con 0x7f8ae01074d0 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.054+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/3696649489 --> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8ae00753f0 con 0x7f8ac8039450 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.059+0000 7f8adcff9700 1 -- 172.21.15.31:0/3696649489 <== mgr.14116 v2:172.21.15.31:6800/3047311576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ae00753f0 con 0x7f8ac8039450 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.061+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/3696649489 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8ac8039450 msgr2=0x7f8ac803b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.061+0000 7f8ae5e0e700 1 --2- 172.21.15.31:0/3696649489 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8ac8039450 0x7f8ac803b910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0006fd0 tx=0x7f8ad0006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.061+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/3696649489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 msgr2=0x7f8ae019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.061+0000 7f8ae5e0e700 1 --2- 172.21.15.31:0/3696649489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae019a250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400bfd0 tx=0x7f8ad4003ff0 comp rx=0 tx=0).stop 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.061+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/3696649489 shutdown_connections 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.061+0000 7f8ae5e0e700 1 --2- 172.21.15.31:0/3696649489 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f8ac8039450 0x7f8ac803b910 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.061+0000 7f8ae5e0e700 1 --2- 172.21.15.31:0/3696649489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae01074d0 0x7f8ae019a250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.061+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/3696649489 >> 172.21.15.31:0/3696649489 conn(0x7f8ae0102ab0 msgr2=0x7f8ae0190890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.061+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/3696649489 shutdown_connections 2024-04-25T13:10:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.061+0000 7f8ae5e0e700 1 -- 172.21.15.31:0/3696649489 wait complete. 2024-04-25T13:10:07.582 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:07 smithi031 ceph-mon[29457]: [25/Apr/2024:13:10:05] ENGINE Bus STARTING 2024-04-25T13:10:07.582 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:07 smithi031 ceph-mon[29457]: [25/Apr/2024:13:10:05] ENGINE Serving on https://172.21.15.31:7150 2024-04-25T13:10:07.582 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:07 smithi031 ceph-mon[29457]: [25/Apr/2024:13:10:06] ENGINE Serving on http://172.21.15.31:8765 2024-04-25T13:10:07.582 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:07 smithi031 ceph-mon[29457]: [25/Apr/2024:13:10:06] ENGINE Bus STARTED 2024-04-25T13:10:07.583 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:07 smithi031 ceph-mon[29457]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:07.583 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:07 smithi031 ceph-mon[29457]: mgrmap e8: x(active, since 2s) 2024-04-25T13:10:07.849 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T13:10:07.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.996+0000 7fa050ba3700 1 Processor -- start 2024-04-25T13:10:07.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.996+0000 7fa050ba3700 1 -- start start 2024-04-25T13:10:07.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.997+0000 7fa050ba3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c1089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:07.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.997+0000 7fa050ba3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c108fa0 con 0x7fa04c1085b0 2024-04-25T13:10:07.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.997+0000 7fa04a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c1089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:07.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.997+0000 7fa04a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c1089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56824/0 (socket says 172.21.15.31:56824) 2024-04-25T13:10:07.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.997+0000 7fa04a59c700 1 -- 172.21.15.31:0/2116428549 learned_addr learned my addr 172.21.15.31:0/2116428549 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:07.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.997+0000 7fa04a59c700 1 -- 172.21.15.31:0/2116428549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c109760 con 0x7fa04c1085b0 2024-04-25T13:10:07.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.997+0000 7fa04a59c700 1 --2- 172.21.15.31:0/2116428549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c1089d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0098a0 tx=0x7fa03c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c38ca39b811965 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:07.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.998+0000 7fa04959a700 1 -- 172.21.15.31:0/2116428549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa03c004160 con 0x7fa04c1085b0 2024-04-25T13:10:07.850 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.998+0000 7fa04959a700 1 -- 172.21.15.31:0/2116428549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa03c004300 con 0x7fa04c1085b0 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.998+0000 7fa04959a700 1 -- 172.21.15.31:0/2116428549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa03c011af0 con 0x7fa04c1085b0 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.998+0000 7fa050ba3700 1 -- 172.21.15.31:0/2116428549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 msgr2=0x7fa04c1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.998+0000 7fa050ba3700 1 --2- 172.21.15.31:0/2116428549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c1089d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0098a0 tx=0x7fa03c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.998+0000 7fa050ba3700 1 -- 172.21.15.31:0/2116428549 shutdown_connections 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.998+0000 7fa050ba3700 1 --2- 172.21.15.31:0/2116428549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c1089d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.998+0000 7fa050ba3700 1 -- 172.21.15.31:0/2116428549 >> 172.21.15.31:0/2116428549 conn(0x7fa04c103b70 msgr2=0x7fa04c105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.998+0000 7fa050ba3700 1 -- 172.21.15.31:0/2116428549 shutdown_connections 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.998+0000 7fa050ba3700 1 -- 172.21.15.31:0/2116428549 wait complete. 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.999+0000 7fa050ba3700 1 Processor -- start 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.999+0000 7fa050ba3700 1 -- start start 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.999+0000 7fa050ba3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.999+0000 7fa050ba3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c10c990 con 0x7fa04c1085b0 2024-04-25T13:10:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:06.999+0000 7fa04a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.000+0000 7fa04a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56826/0 (socket says 172.21.15.31:56826) 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.000+0000 7fa04a59c700 1 -- 172.21.15.31:0/2868677033 learned_addr learned my addr 172.21.15.31:0/2868677033 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.000+0000 7fa04a59c700 1 -- 172.21.15.31:0/2868677033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c19ca00 con 0x7fa04c1085b0 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.000+0000 7fa04a59c700 1 --2- 172.21.15.31:0/2868677033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c19c4c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa03c000c00 tx=0x7fa03c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.000+0000 7fa03b7fe700 1 -- 172.21.15.31:0/2868677033 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa03c003b20 con 0x7fa04c1085b0 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.000+0000 7fa03b7fe700 1 -- 172.21.15.31:0/2868677033 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa03c01a8c0 con 0x7fa04c1085b0 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.000+0000 7fa03b7fe700 1 -- 172.21.15.31:0/2868677033 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa03c024940 con 0x7fa04c1085b0 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.000+0000 7fa050ba3700 1 -- 172.21.15.31:0/2868677033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c19cc90 con 0x7fa04c1085b0 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.000+0000 7fa050ba3700 1 -- 172.21.15.31:0/2868677033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c19f980 con 0x7fa04c1085b0 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.001+0000 7fa03b7fe700 1 -- 172.21.15.31:0/2868677033 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa03c022020 con 0x7fa04c1085b0 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.001+0000 7fa050ba3700 1 -- 172.21.15.31:0/2868677033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c10cb10 con 0x7fa04c1085b0 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.001+0000 7fa03b7fe700 1 --2- 172.21.15.31:0/2868677033 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fa034039600 0x7fa03403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.001+0000 7fa03b7fe700 1 -- 172.21.15.31:0/2868677033 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa03c04da10 con 0x7fa04c1085b0 2024-04-25T13:10:07.852 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.001+0000 7fa049d9b700 1 --2- 172.21.15.31:0/2868677033 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fa034039600 0x7fa03403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.002+0000 7fa049d9b700 1 --2- 172.21.15.31:0/2868677033 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fa034039600 0x7fa03403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa040006fd0 tx=0x7fa040006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.007+0000 7fa03b7fe700 1 -- 172.21.15.31:0/2868677033 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa03c021d80 con 0x7fa04c1085b0 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.214+0000 7fa050ba3700 1 -- 172.21.15.31:0/2868677033 --> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa04c104e10 con 0x7fa034039600 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.215+0000 7fa03b7fe700 1 -- 172.21.15.31:0/2868677033 <== mgr.14116 v2:172.21.15.31:6800/3047311576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa04c104e10 con 0x7fa034039600 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.217+0000 7fa050ba3700 1 -- 172.21.15.31:0/2868677033 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fa034039600 msgr2=0x7fa03403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.217+0000 7fa050ba3700 1 --2- 172.21.15.31:0/2868677033 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fa034039600 0x7fa03403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa040006fd0 tx=0x7fa040006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.217+0000 7fa050ba3700 1 -- 172.21.15.31:0/2868677033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 msgr2=0x7fa04c19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.217+0000 7fa050ba3700 1 --2- 172.21.15.31:0/2868677033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c19c4c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa03c000c00 tx=0x7fa03c011b70 comp rx=0 tx=0).stop 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.217+0000 7fa050ba3700 1 -- 172.21.15.31:0/2868677033 shutdown_connections 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.217+0000 7fa050ba3700 1 --2- 172.21.15.31:0/2868677033 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fa034039600 0x7fa03403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.217+0000 7fa050ba3700 1 --2- 172.21.15.31:0/2868677033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa04c1085b0 0x7fa04c19c4c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.217+0000 7fa050ba3700 1 -- 172.21.15.31:0/2868677033 >> 172.21.15.31:0/2868677033 conn(0x7fa04c103b70 msgr2=0x7fa04c104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.217+0000 7fa050ba3700 1 -- 172.21.15.31:0/2868677033 shutdown_connections 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:07.217+0000 7fa050ba3700 1 -- 172.21.15.31:0/2868677033 wait complete. 2024-04-25T13:10:07.853 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2024-04-25T13:10:08.469 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-mon[29457]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:08.725 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: Generating public/private rsa key pair. 2024-04-25T13:10:08.725 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: Your identification has been saved in /tmp/tmpwzhstqas/key. 2024-04-25T13:10:08.725 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: Your public key has been saved in /tmp/tmpwzhstqas/key.pub. 2024-04-25T13:10:08.725 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: The key fingerprint is: 2024-04-25T13:10:08.725 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: SHA256:I78DbIR3/QokahjZaV+19g0kutSkLO5ljX00di0CgyA ceph-e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:10:08.725 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: The key's randomart image is: 2024-04-25T13:10:08.725 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: +---[RSA 3072]----+ 2024-04-25T13:10:08.725 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: | E .. . | 2024-04-25T13:10:08.725 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: | . . * . | 2024-04-25T13:10:08.726 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: | o o . B * . | 2024-04-25T13:10:08.726 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: | o = = O = * o .| 2024-04-25T13:10:08.726 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: | + B.BS* = * . | 2024-04-25T13:10:08.726 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: | . o *o*.o + . | 2024-04-25T13:10:08.726 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: | . o +.. o | 2024-04-25T13:10:08.726 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: | . ... | 2024-04-25T13:10:08.726 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: | .. | 2024-04-25T13:10:08.726 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: +----[SHA256]-----+ 2024-04-25T13:10:09.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.248+0000 7fb62ed40700 1 Processor -- start 2024-04-25T13:10:09.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.248+0000 7fb62ed40700 1 -- start start 2024-04-25T13:10:09.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.248+0000 7fb62ed40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb628108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:09.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.248+0000 7fb62ed40700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb628108ff0 con 0x7fb628108600 2024-04-25T13:10:09.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.248+0000 7fb62cadc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb628108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.249+0000 7fb62cadc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb628108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56828/0 (socket says 172.21.15.31:56828) 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.249+0000 7fb62cadc700 1 -- 172.21.15.31:0/3774082309 learned_addr learned my addr 172.21.15.31:0/3774082309 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.249+0000 7fb62cadc700 1 -- 172.21.15.31:0/3774082309 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6281097b0 con 0x7fb628108600 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.249+0000 7fb62cadc700 1 --2- 172.21.15.31:0/3774082309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb628108a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0098a0 tx=0x7fb61c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4629cf27ada530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.249+0000 7fb6277fe700 1 -- 172.21.15.31:0/3774082309 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb61c004160 con 0x7fb628108600 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.249+0000 7fb6277fe700 1 -- 172.21.15.31:0/3774082309 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb61c004300 con 0x7fb628108600 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.249+0000 7fb6277fe700 1 -- 172.21.15.31:0/3774082309 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb61c011af0 con 0x7fb628108600 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.250+0000 7fb62ed40700 1 -- 172.21.15.31:0/3774082309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 msgr2=0x7fb628108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.250+0000 7fb62ed40700 1 --2- 172.21.15.31:0/3774082309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb628108a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0098a0 tx=0x7fb61c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.250+0000 7fb62ed40700 1 -- 172.21.15.31:0/3774082309 shutdown_connections 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.250+0000 7fb62ed40700 1 --2- 172.21.15.31:0/3774082309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb628108a20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.250+0000 7fb62ed40700 1 -- 172.21.15.31:0/3774082309 >> 172.21.15.31:0/3774082309 conn(0x7fb628103bc0 msgr2=0x7fb628105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.250+0000 7fb62ed40700 1 -- 172.21.15.31:0/3774082309 shutdown_connections 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.250+0000 7fb62ed40700 1 -- 172.21.15.31:0/3774082309 wait complete. 2024-04-25T13:10:09.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.250+0000 7fb62ed40700 1 Processor -- start 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.251+0000 7fb62ed40700 1 -- start start 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.251+0000 7fb62ed40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb62819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.251+0000 7fb62ed40700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb62810c9e0 con 0x7fb628108600 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.251+0000 7fb62cadc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb62819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.251+0000 7fb62cadc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb62819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56838/0 (socket says 172.21.15.31:56838) 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.251+0000 7fb62cadc700 1 -- 172.21.15.31:0/1082429341 learned_addr learned my addr 172.21.15.31:0/1082429341 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.251+0000 7fb62cadc700 1 -- 172.21.15.31:0/1082429341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62819ec10 con 0x7fb628108600 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.251+0000 7fb62cadc700 1 --2- 172.21.15.31:0/1082429341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb62819e6d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb61c003f40 tx=0x7fb61c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.251+0000 7fb625ffb700 1 -- 172.21.15.31:0/1082429341 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb61c0038b0 con 0x7fb628108600 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.251+0000 7fb625ffb700 1 -- 172.21.15.31:0/1082429341 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb61c003a50 con 0x7fb628108600 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.251+0000 7fb62ed40700 1 -- 172.21.15.31:0/1082429341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62819eea0 con 0x7fb628108600 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.252+0000 7fb625ffb700 1 -- 172.21.15.31:0/1082429341 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb61c01a5c0 con 0x7fb628108600 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.252+0000 7fb62ed40700 1 -- 172.21.15.31:0/1082429341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6281a1b90 con 0x7fb628108600 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.253+0000 7fb625ffb700 1 -- 172.21.15.31:0/1082429341 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb61c01a870 con 0x7fb628108600 2024-04-25T13:10:09.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.253+0000 7fb62ed40700 1 -- 172.21.15.31:0/1082429341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb62810cb60 con 0x7fb628108600 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.253+0000 7fb625ffb700 1 --2- 172.21.15.31:0/1082429341 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fb6180395b0 0x7fb61803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.253+0000 7fb625ffb700 1 -- 172.21.15.31:0/1082429341 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb61c04da90 con 0x7fb628108600 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.253+0000 7fb627fff700 1 --2- 172.21.15.31:0/1082429341 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fb6180395b0 0x7fb61803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.254+0000 7fb627fff700 1 --2- 172.21.15.31:0/1082429341 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fb6180395b0 0x7fb61803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb614006fd0 tx=0x7fb614006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.258+0000 7fb625ffb700 1 -- 172.21.15.31:0/1082429341 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb61c02d3f0 con 0x7fb628108600 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.468+0000 7fb62ed40700 1 -- 172.21.15.31:0/1082429341 --> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fb628104d60 con 0x7fb6180395b0 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.711+0000 7fb625ffb700 1 -- 172.21.15.31:0/1082429341 <== mgr.14116 v2:172.21.15.31:6800/3047311576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb628104d60 con 0x7fb6180395b0 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.713+0000 7fb62ed40700 1 -- 172.21.15.31:0/1082429341 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fb6180395b0 msgr2=0x7fb61803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.713+0000 7fb62ed40700 1 --2- 172.21.15.31:0/1082429341 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fb6180395b0 0x7fb61803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb614006fd0 tx=0x7fb614006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.713+0000 7fb62ed40700 1 -- 172.21.15.31:0/1082429341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 msgr2=0x7fb62819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.713+0000 7fb62ed40700 1 --2- 172.21.15.31:0/1082429341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb62819e6d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb61c003f40 tx=0x7fb61c011890 comp rx=0 tx=0).stop 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.713+0000 7fb62ed40700 1 -- 172.21.15.31:0/1082429341 shutdown_connections 2024-04-25T13:10:09.350 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.713+0000 7fb62ed40700 1 --2- 172.21.15.31:0/1082429341 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fb6180395b0 0x7fb61803ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:09.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.713+0000 7fb62ed40700 1 --2- 172.21.15.31:0/1082429341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb628108600 0x7fb62819e6d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:09.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.713+0000 7fb62ed40700 1 -- 172.21.15.31:0/1082429341 >> 172.21.15.31:0/1082429341 conn(0x7fb628103bc0 msgr2=0x7fb628104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:09.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.713+0000 7fb62ed40700 1 -- 172.21.15.31:0/1082429341 shutdown_connections 2024-04-25T13:10:09.351 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:08.713+0000 7fb62ed40700 1 -- 172.21.15.31:0/1082429341 wait complete. 2024-04-25T13:10:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:09 smithi031 ceph-mon[29457]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:09 smithi031 ceph-mon[29457]: Generating ssh key... 2024-04-25T13:10:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:09 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:09 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd21Kz63tEiVZqeHg2TtSIOO6/5WQ4oj7DOv9bHDL/XMY6D16abVmSEXoymVZqjP1yBG/NaJX0Brcfo+PCwIzHR4j92uHXqJoNDqaSGtFK2FevzzfdZwzUUo6VKuO1036ak9Sy5ly9Hg7fOUtGiGHdXdr/XfVXyrNh+Rc1WiZqItdmtL/eeR1b71UHHSXJLGAP2b1m6Zmbw/M/Gec0oCxKwX3TOq/dmrKpfJaVDWE4qpGRa4E2MRpTb+0tf0c1UkKChlOYWgfwMZe/NQsHoznSbNxaS1EZVjuArk/279ggX23RcjtMDpS8QgCV5/zDAU122VJHQR0Y0pw9DcWgy20ABD3DGA3TgAVsUG4Q4RjMrY3rJd4lfp/fMFhfkVOFimCyZHq/m7jV03lWmn0aXQxl9kGMnQzNUpRp6mZK1UObkyx9niHY231WN3poji7BFeJgqBR1rdXtc33GI5NnRcOAPOi6yMTlCKp1yWj1w3aw3qcE4QRCCWvjCkqnUHVOB80= ceph-e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:10:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.769+0000 7fc510067700 1 Processor -- start 2024-04-25T13:10:10.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.769+0000 7fc510067700 1 -- start start 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.769+0000 7fc510067700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50810b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.769+0000 7fc510067700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc50807be30 con 0x7fc5081090c0 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.769+0000 7fc50de03700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50810b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.769+0000 7fc50de03700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50810b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56850/0 (socket says 172.21.15.31:56850) 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.769+0000 7fc50de03700 1 -- 172.21.15.31:0/3634247580 learned_addr learned my addr 172.21.15.31:0/3634247580 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.770+0000 7fc50de03700 1 -- 172.21.15.31:0/3634247580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50810b9f0 con 0x7fc5081090c0 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.770+0000 7fc50de03700 1 --2- 172.21.15.31:0/3634247580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50810b4b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc5040098a0 tx=0x7fc504009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346bf578e5d2990c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.770+0000 7fc50ce01700 1 -- 172.21.15.31:0/3634247580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc504004160 con 0x7fc5081090c0 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.770+0000 7fc50ce01700 1 -- 172.21.15.31:0/3634247580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc504004300 con 0x7fc5081090c0 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.770+0000 7fc50ce01700 1 -- 172.21.15.31:0/3634247580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc504011af0 con 0x7fc5081090c0 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.771+0000 7fc510067700 1 -- 172.21.15.31:0/3634247580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 msgr2=0x7fc50810b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.771+0000 7fc510067700 1 --2- 172.21.15.31:0/3634247580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50810b4b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc5040098a0 tx=0x7fc504009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.771+0000 7fc510067700 1 -- 172.21.15.31:0/3634247580 shutdown_connections 2024-04-25T13:10:10.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.771+0000 7fc510067700 1 --2- 172.21.15.31:0/3634247580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50810b4b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.771+0000 7fc510067700 1 -- 172.21.15.31:0/3634247580 >> 172.21.15.31:0/3634247580 conn(0x7fc508102a90 msgr2=0x7fc508104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.771+0000 7fc510067700 1 -- 172.21.15.31:0/3634247580 shutdown_connections 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.771+0000 7fc510067700 1 -- 172.21.15.31:0/3634247580 wait complete. 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.772+0000 7fc510067700 1 Processor -- start 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.772+0000 7fc510067700 1 -- start start 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.772+0000 7fc510067700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.772+0000 7fc510067700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc50810c8f0 con 0x7fc5081090c0 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.772+0000 7fc50de03700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.772+0000 7fc50de03700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56856/0 (socket says 172.21.15.31:56856) 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.772+0000 7fc50de03700 1 -- 172.21.15.31:0/3393098102 learned_addr learned my addr 172.21.15.31:0/3393098102 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.772+0000 7fc50de03700 1 -- 172.21.15.31:0/3393098102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50819ebb0 con 0x7fc5081090c0 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.772+0000 7fc50de03700 1 --2- 172.21.15.31:0/3393098102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50819e670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc504011ac0 tx=0x7fc504011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.773+0000 7fc4feffd700 1 -- 172.21.15.31:0/3393098102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc504003b20 con 0x7fc5081090c0 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.773+0000 7fc4feffd700 1 -- 172.21.15.31:0/3393098102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc50401a8c0 con 0x7fc5081090c0 2024-04-25T13:10:10.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.773+0000 7fc4feffd700 1 -- 172.21.15.31:0/3393098102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5040248d0 con 0x7fc5081090c0 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.773+0000 7fc510067700 1 -- 172.21.15.31:0/3393098102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50819ee40 con 0x7fc5081090c0 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.773+0000 7fc510067700 1 -- 172.21.15.31:0/3393098102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5081a1b30 con 0x7fc5081090c0 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.774+0000 7fc4feffd700 1 -- 172.21.15.31:0/3393098102 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc504022020 con 0x7fc5081090c0 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.774+0000 7fc510067700 1 -- 172.21.15.31:0/3393098102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5080640a0 con 0x7fc5081090c0 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.774+0000 7fc4feffd700 1 --2- 172.21.15.31:0/3393098102 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4f40395b0 0x7fc4f403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.774+0000 7fc4feffd700 1 -- 172.21.15.31:0/3393098102 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc50404d8a0 con 0x7fc5081090c0 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.774+0000 7fc50d602700 1 --2- 172.21.15.31:0/3393098102 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4f40395b0 0x7fc4f403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.775+0000 7fc50d602700 1 --2- 172.21.15.31:0/3393098102 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4f40395b0 0x7fc4f403ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8006fd0 tx=0x7fc4f8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.779+0000 7fc4feffd700 1 -- 172.21.15.31:0/3393098102 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc50400f2e0 con 0x7fc5081090c0 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.991+0000 7fc510067700 1 -- 172.21.15.31:0/3393098102 --> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fc508103c10 con 0x7fc4f40395b0 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.992+0000 7fc4feffd700 1 -- 172.21.15.31:0/3393098102 <== mgr.14116 v2:172.21.15.31:6800/3047311576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fc508103c10 con 0x7fc4f40395b0 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.994+0000 7fc510067700 1 -- 172.21.15.31:0/3393098102 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4f40395b0 msgr2=0x7fc4f403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:10.566 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.994+0000 7fc510067700 1 --2- 172.21.15.31:0/3393098102 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4f40395b0 0x7fc4f403ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8006fd0 tx=0x7fc4f8006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.994+0000 7fc510067700 1 -- 172.21.15.31:0/3393098102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 msgr2=0x7fc50819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.994+0000 7fc510067700 1 --2- 172.21.15.31:0/3393098102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50819e670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc504011ac0 tx=0x7fc504011ba0 comp rx=0 tx=0).stop 2024-04-25T13:10:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.994+0000 7fc510067700 1 -- 172.21.15.31:0/3393098102 shutdown_connections 2024-04-25T13:10:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.994+0000 7fc510067700 1 --2- 172.21.15.31:0/3393098102 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4f40395b0 0x7fc4f403ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.994+0000 7fc510067700 1 --2- 172.21.15.31:0/3393098102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5081090c0 0x7fc50819e670 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.994+0000 7fc510067700 1 -- 172.21.15.31:0/3393098102 >> 172.21.15.31:0/3393098102 conn(0x7fc508102a90 msgr2=0x7fc508103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.995+0000 7fc510067700 1 -- 172.21.15.31:0/3393098102 shutdown_connections 2024-04-25T13:10:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:09.995+0000 7fc510067700 1 -- 172.21.15.31:0/3393098102 wait complete. 2024-04-25T13:10:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:10:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T13:10:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2024-04-25T13:10:10.855 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:10 smithi031 ceph-mon[29457]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:12.080 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:11 smithi031 ceph-mon[29457]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:13.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:12 smithi031 ceph-mon[29457]: Deploying cephadm binary to smithi031 2024-04-25T13:10:15.976 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2024-04-25T13:10:15.976 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.968+0000 7f08b808f700 1 Processor -- start 2024-04-25T13:10:15.976 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.968+0000 7f08b808f700 1 -- start start 2024-04-25T13:10:15.976 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.968+0000 7f08b808f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b010b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.968+0000 7f08b808f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f08b007be30 con 0x7f08b01090a0 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.968+0000 7f08b5e2b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b010b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.968+0000 7f08b5e2b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b010b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56872/0 (socket says 172.21.15.31:56872) 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.969+0000 7f08b5e2b700 1 -- 172.21.15.31:0/2451121942 learned_addr learned my addr 172.21.15.31:0/2451121942 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.969+0000 7f08b5e2b700 1 -- 172.21.15.31:0/2451121942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b010b9d0 con 0x7f08b01090a0 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.969+0000 7f08b5e2b700 1 --2- 172.21.15.31:0/2451121942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b010b490 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0098a0 tx=0x7f08ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d304f2d387b3d494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.969+0000 7f08b4e29700 1 -- 172.21.15.31:0/2451121942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08ac004160 con 0x7f08b01090a0 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.969+0000 7f08b4e29700 1 -- 172.21.15.31:0/2451121942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f08ac004300 con 0x7f08b01090a0 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.969+0000 7f08b4e29700 1 -- 172.21.15.31:0/2451121942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08ac011af0 con 0x7f08b01090a0 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.970+0000 7f08b808f700 1 -- 172.21.15.31:0/2451121942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 msgr2=0x7f08b010b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.970+0000 7f08b808f700 1 --2- 172.21.15.31:0/2451121942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b010b490 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0098a0 tx=0x7f08ac009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.970+0000 7f08b808f700 1 -- 172.21.15.31:0/2451121942 shutdown_connections 2024-04-25T13:10:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.970+0000 7f08b808f700 1 --2- 172.21.15.31:0/2451121942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b010b490 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.970+0000 7f08b808f700 1 -- 172.21.15.31:0/2451121942 >> 172.21.15.31:0/2451121942 conn(0x7f08b0102ad0 msgr2=0x7f08b0104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.970+0000 7f08b808f700 1 -- 172.21.15.31:0/2451121942 shutdown_connections 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.970+0000 7f08b808f700 1 -- 172.21.15.31:0/2451121942 wait complete. 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.970+0000 7f08b808f700 1 Processor -- start 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.971+0000 7f08b808f700 1 -- start start 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.971+0000 7f08b808f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b0108920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.971+0000 7f08b808f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f08b010c8d0 con 0x7f08b01090a0 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.971+0000 7f08b5e2b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b0108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.971+0000 7f08b5e2b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b0108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56882/0 (socket says 172.21.15.31:56882) 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.971+0000 7f08b5e2b700 1 -- 172.21.15.31:0/302687720 learned_addr learned my addr 172.21.15.31:0/302687720 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.971+0000 7f08b5e2b700 1 -- 172.21.15.31:0/302687720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b0108e60 con 0x7f08b01090a0 2024-04-25T13:10:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.971+0000 7f08b5e2b700 1 --2- 172.21.15.31:0/302687720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b0108920 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f08ac011ac0 tx=0x7f08ac011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.971+0000 7f08a6ffd700 1 -- 172.21.15.31:0/302687720 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08ac003b20 con 0x7f08b01090a0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.972+0000 7f08a6ffd700 1 -- 172.21.15.31:0/302687720 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f08ac01a8c0 con 0x7f08b01090a0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.972+0000 7f08b808f700 1 -- 172.21.15.31:0/302687720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b0107050 con 0x7f08b01090a0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.972+0000 7f08a6ffd700 1 -- 172.21.15.31:0/302687720 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08ac0248d0 con 0x7f08b01090a0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.972+0000 7f08b808f700 1 -- 172.21.15.31:0/302687720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b0107590 con 0x7f08b01090a0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.972+0000 7f08a6ffd700 1 -- 172.21.15.31:0/302687720 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f08ac022020 con 0x7f08b01090a0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.973+0000 7f08b808f700 1 -- 172.21.15.31:0/302687720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b00640a0 con 0x7f08b01090a0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.973+0000 7f08a6ffd700 1 --2- 172.21.15.31:0/302687720 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f089c039600 0x7f089c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.973+0000 7f08a6ffd700 1 -- 172.21.15.31:0/302687720 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f08ac04d8a0 con 0x7f08b01090a0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.973+0000 7f08b562a700 1 --2- 172.21.15.31:0/302687720 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f089c039600 0x7f089c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.973+0000 7f08b562a700 1 --2- 172.21.15.31:0/302687720 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f089c039600 0x7f089c03bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f08a0006fd0 tx=0x7f08a0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:10.978+0000 7f08a6ffd700 1 -- 172.21.15.31:0/302687720 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08ac0172b0 con 0x7f08b01090a0 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:11.190+0000 7f08b808f700 1 -- 172.21.15.31:0/302687720 --> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}) -- 0x7f08b0103cd0 con 0x7f089c039600 2024-04-25T13:10:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:15.385+0000 7f08a6ffd700 1 -- 172.21.15.31:0/302687720 <== mgr.14116 v2:172.21.15.31:6800/3047311576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f08b0103cd0 con 0x7f089c039600 2024-04-25T13:10:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:15.388+0000 7f08b808f700 1 -- 172.21.15.31:0/302687720 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f089c039600 msgr2=0x7f089c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:15.388+0000 7f08b808f700 1 --2- 172.21.15.31:0/302687720 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f089c039600 0x7f089c03bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f08a0006fd0 tx=0x7f08a0006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:15.388+0000 7f08b808f700 1 -- 172.21.15.31:0/302687720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 msgr2=0x7f08b0108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:15.388+0000 7f08b808f700 1 --2- 172.21.15.31:0/302687720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b0108920 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f08ac011ac0 tx=0x7f08ac011ba0 comp rx=0 tx=0).stop 2024-04-25T13:10:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:15.388+0000 7f08b808f700 1 -- 172.21.15.31:0/302687720 shutdown_connections 2024-04-25T13:10:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:15.388+0000 7f08b808f700 1 --2- 172.21.15.31:0/302687720 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f089c039600 0x7f089c03bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:15.388+0000 7f08b808f700 1 --2- 172.21.15.31:0/302687720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08b01090a0 0x7f08b0108920 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:15.388+0000 7f08b808f700 1 -- 172.21.15.31:0/302687720 >> 172.21.15.31:0/302687720 conn(0x7f08b0102ad0 msgr2=0x7f08b01035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:15.388+0000 7f08b808f700 1 -- 172.21.15.31:0/302687720 shutdown_connections 2024-04-25T13:10:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:15.388+0000 7f08b808f700 1 -- 172.21.15.31:0/302687720 wait complete. 2024-04-25T13:10:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mon service... 2024-04-25T13:10:16.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:16 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:16.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:16 smithi031 ceph-mon[29457]: Added host smithi031 2024-04-25T13:10:16.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:16 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:17.232 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T13:10:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.418+0000 7fafb4b8f700 1 Processor -- start 2024-04-25T13:10:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.418+0000 7fafb4b8f700 1 -- start start 2024-04-25T13:10:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.418+0000 7fafb4b8f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb01072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.418+0000 7fafb4b8f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fafb0063d00 con 0x7fafb0106e90 2024-04-25T13:10:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.419+0000 7fafae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb01072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.419+0000 7fafae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb01072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34088/0 (socket says 172.21.15.31:34088) 2024-04-25T13:10:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.419+0000 7fafae59c700 1 -- 172.21.15.31:0/3403014000 learned_addr learned my addr 172.21.15.31:0/3403014000 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.419+0000 7fafae59c700 1 -- 172.21.15.31:0/3403014000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb01077f0 con 0x7fafb0106e90 2024-04-25T13:10:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.419+0000 7fafae59c700 1 --2- 172.21.15.31:0/3403014000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb01072b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fafa00098a0 tx=0x7fafa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f2305d64c2913d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.419+0000 7fafad59a700 1 -- 172.21.15.31:0/3403014000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafa0004160 con 0x7fafb0106e90 2024-04-25T13:10:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.419+0000 7fafad59a700 1 -- 172.21.15.31:0/3403014000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fafa0004300 con 0x7fafb0106e90 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.419+0000 7fafad59a700 1 -- 172.21.15.31:0/3403014000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafa0011af0 con 0x7fafb0106e90 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.420+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3403014000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 msgr2=0x7fafb01072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.420+0000 7fafb4b8f700 1 --2- 172.21.15.31:0/3403014000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb01072b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fafa00098a0 tx=0x7fafa0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.420+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3403014000 shutdown_connections 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.420+0000 7fafb4b8f700 1 --2- 172.21.15.31:0/3403014000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb01072b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.420+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3403014000 >> 172.21.15.31:0/3403014000 conn(0x7fafb0102ab0 msgr2=0x7fafb0104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.420+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3403014000 shutdown_connections 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.420+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3403014000 wait complete. 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.421+0000 7fafb4b8f700 1 Processor -- start 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.421+0000 7fafb4b8f700 1 -- start start 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.421+0000 7fafb4b8f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.421+0000 7fafb4b8f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fafb01087b0 con 0x7fafb0106e90 2024-04-25T13:10:17.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.421+0000 7fafae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.421+0000 7fafae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34100/0 (socket says 172.21.15.31:34100) 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.421+0000 7fafae59c700 1 -- 172.21.15.31:0/3992958568 learned_addr learned my addr 172.21.15.31:0/3992958568 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.421+0000 7fafae59c700 1 -- 172.21.15.31:0/3992958568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb019ebf0 con 0x7fafb0106e90 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.422+0000 7fafae59c700 1 --2- 172.21.15.31:0/3992958568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb019e6b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fafa0011ac0 tx=0x7fafa0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.422+0000 7faf9f7fe700 1 -- 172.21.15.31:0/3992958568 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafa0003b20 con 0x7fafb0106e90 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.422+0000 7faf9f7fe700 1 -- 172.21.15.31:0/3992958568 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fafa001a8c0 con 0x7fafb0106e90 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.422+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3992958568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb019ee80 con 0x7fafb0106e90 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.422+0000 7faf9f7fe700 1 -- 172.21.15.31:0/3992958568 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafa00248d0 con 0x7fafb0106e90 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.422+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3992958568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb01a1b70 con 0x7fafb0106e90 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.423+0000 7faf9f7fe700 1 -- 172.21.15.31:0/3992958568 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fafa0022020 con 0x7fafb0106e90 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.423+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3992958568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb00640a0 con 0x7fafb0106e90 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.423+0000 7faf9f7fe700 1 --2- 172.21.15.31:0/3992958568 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7faf98039600 0x7faf9803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.423+0000 7faf9f7fe700 1 -- 172.21.15.31:0/3992958568 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fafa004d990 con 0x7fafb0106e90 2024-04-25T13:10:17.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.423+0000 7fafadd9b700 1 --2- 172.21.15.31:0/3992958568 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7faf98039600 0x7faf9803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.424+0000 7fafadd9b700 1 --2- 172.21.15.31:0/3992958568 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7faf98039600 0x7faf9803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fafa4006fd0 tx=0x7fafa4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.428+0000 7faf9f7fe700 1 -- 172.21.15.31:0/3992958568 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafa001ade0 con 0x7fafb0106e90 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.655+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3992958568 --> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fafb002e260 con 0x7faf98039600 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.660+0000 7faf9f7fe700 1 -- 172.21.15.31:0/3992958568 <== mgr.14116 v2:172.21.15.31:6800/3047311576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fafb002e260 con 0x7faf98039600 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.662+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3992958568 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7faf98039600 msgr2=0x7faf9803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.662+0000 7fafb4b8f700 1 --2- 172.21.15.31:0/3992958568 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7faf98039600 0x7faf9803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fafa4006fd0 tx=0x7fafa4006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.662+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3992958568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 msgr2=0x7fafb019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.662+0000 7fafb4b8f700 1 --2- 172.21.15.31:0/3992958568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb019e6b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fafa0011ac0 tx=0x7fafa0011ba0 comp rx=0 tx=0).stop 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.662+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3992958568 shutdown_connections 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.662+0000 7fafb4b8f700 1 --2- 172.21.15.31:0/3992958568 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7faf98039600 0x7faf9803bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.662+0000 7fafb4b8f700 1 --2- 172.21.15.31:0/3992958568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb0106e90 0x7fafb019e6b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.662+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3992958568 >> 172.21.15.31:0/3992958568 conn(0x7fafb0102ab0 msgr2=0x7fafb0195350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:17.237 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.662+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3992958568 shutdown_connections 2024-04-25T13:10:17.237 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:16.662+0000 7fafb4b8f700 1 -- 172.21.15.31:0/3992958568 wait complete. 2024-04-25T13:10:17.237 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mgr service... 2024-04-25T13:10:17.917 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:17 smithi031 ceph-mon[29457]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:17.918 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:17 smithi031 ceph-mon[29457]: Saving service mon spec with placement count:5 2024-04-25T13:10:17.918 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:17 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:18.525 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T13:10:18.525 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.676+0000 7fc446ca9700 1 Processor -- start 2024-04-25T13:10:18.525 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.676+0000 7fc446ca9700 1 -- start start 2024-04-25T13:10:18.525 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.677+0000 7fc446ca9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc4401072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.677+0000 7fc446ca9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc440063ce0 con 0x7fc440106eb0 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.677+0000 7fc444a45700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc4401072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.677+0000 7fc444a45700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc4401072d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34104/0 (socket says 172.21.15.31:34104) 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.677+0000 7fc444a45700 1 -- 172.21.15.31:0/2269387626 learned_addr learned my addr 172.21.15.31:0/2269387626 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.677+0000 7fc444a45700 1 -- 172.21.15.31:0/2269387626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc440107810 con 0x7fc440106eb0 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.677+0000 7fc444a45700 1 --2- 172.21.15.31:0/2269387626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc4401072d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4340098a0 tx=0x7fc434009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e09e7f900934281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.677+0000 7fc43f7fe700 1 -- 172.21.15.31:0/2269387626 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc434004160 con 0x7fc440106eb0 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.677+0000 7fc43f7fe700 1 -- 172.21.15.31:0/2269387626 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc434004300 con 0x7fc440106eb0 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.677+0000 7fc43f7fe700 1 -- 172.21.15.31:0/2269387626 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc434011af0 con 0x7fc440106eb0 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.678+0000 7fc446ca9700 1 -- 172.21.15.31:0/2269387626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 msgr2=0x7fc4401072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.678+0000 7fc446ca9700 1 --2- 172.21.15.31:0/2269387626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc4401072d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4340098a0 tx=0x7fc434009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.678+0000 7fc446ca9700 1 -- 172.21.15.31:0/2269387626 shutdown_connections 2024-04-25T13:10:18.526 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.678+0000 7fc446ca9700 1 --2- 172.21.15.31:0/2269387626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc4401072d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.678+0000 7fc446ca9700 1 -- 172.21.15.31:0/2269387626 >> 172.21.15.31:0/2269387626 conn(0x7fc440102ab0 msgr2=0x7fc440104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.678+0000 7fc446ca9700 1 -- 172.21.15.31:0/2269387626 shutdown_connections 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.678+0000 7fc446ca9700 1 -- 172.21.15.31:0/2269387626 wait complete. 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.679+0000 7fc446ca9700 1 Processor -- start 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.679+0000 7fc446ca9700 1 -- start start 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.679+0000 7fc446ca9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc44019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.679+0000 7fc446ca9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401087d0 con 0x7fc440106eb0 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.679+0000 7fc444a45700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc44019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.679+0000 7fc444a45700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc44019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34108/0 (socket says 172.21.15.31:34108) 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.679+0000 7fc444a45700 1 -- 172.21.15.31:0/119929696 learned_addr learned my addr 172.21.15.31:0/119929696 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.679+0000 7fc444a45700 1 -- 172.21.15.31:0/119929696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44019eba0 con 0x7fc440106eb0 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.679+0000 7fc444a45700 1 --2- 172.21.15.31:0/119929696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc44019e660 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc434003f40 tx=0x7fc434011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.679+0000 7fc43dffb700 1 -- 172.21.15.31:0/119929696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4340038b0 con 0x7fc440106eb0 2024-04-25T13:10:18.527 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.679+0000 7fc43dffb700 1 -- 172.21.15.31:0/119929696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc434003a50 con 0x7fc440106eb0 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.680+0000 7fc446ca9700 1 -- 172.21.15.31:0/119929696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44019ee30 con 0x7fc440106eb0 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.680+0000 7fc43dffb700 1 -- 172.21.15.31:0/119929696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc43401a5c0 con 0x7fc440106eb0 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.680+0000 7fc446ca9700 1 -- 172.21.15.31:0/119929696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4401a1b20 con 0x7fc440106eb0 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.681+0000 7fc43dffb700 1 -- 172.21.15.31:0/119929696 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc434024430 con 0x7fc440106eb0 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.681+0000 7fc446ca9700 1 -- 172.21.15.31:0/119929696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4400640a0 con 0x7fc440106eb0 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.681+0000 7fc43dffb700 1 --2- 172.21.15.31:0/119929696 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4300395b0 0x7fc43003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.681+0000 7fc43dffb700 1 -- 172.21.15.31:0/119929696 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc43404dab0 con 0x7fc440106eb0 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.681+0000 7fc43ffff700 1 --2- 172.21.15.31:0/119929696 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4300395b0 0x7fc43003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.681+0000 7fc43ffff700 1 --2- 172.21.15.31:0/119929696 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4300395b0 0x7fc43003ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc42c006fd0 tx=0x7fc42c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.686+0000 7fc43dffb700 1 -- 172.21.15.31:0/119929696 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc43402d970 con 0x7fc440106eb0 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.917+0000 7fc446ca9700 1 -- 172.21.15.31:0/119929696 --> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fc44002e260 con 0x7fc4300395b0 2024-04-25T13:10:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.924+0000 7fc43dffb700 1 -- 172.21.15.31:0/119929696 <== mgr.14116 v2:172.21.15.31:6800/3047311576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc44002e260 con 0x7fc4300395b0 2024-04-25T13:10:18.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.926+0000 7fc446ca9700 1 -- 172.21.15.31:0/119929696 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4300395b0 msgr2=0x7fc43003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:18.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.926+0000 7fc446ca9700 1 --2- 172.21.15.31:0/119929696 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4300395b0 0x7fc43003ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc42c006fd0 tx=0x7fc42c006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:18.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.926+0000 7fc446ca9700 1 -- 172.21.15.31:0/119929696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 msgr2=0x7fc44019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:18.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.926+0000 7fc446ca9700 1 --2- 172.21.15.31:0/119929696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc44019e660 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc434003f40 tx=0x7fc434011890 comp rx=0 tx=0).stop 2024-04-25T13:10:18.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.926+0000 7fc446ca9700 1 -- 172.21.15.31:0/119929696 shutdown_connections 2024-04-25T13:10:18.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.926+0000 7fc446ca9700 1 --2- 172.21.15.31:0/119929696 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fc4300395b0 0x7fc43003ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:18.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.926+0000 7fc446ca9700 1 --2- 172.21.15.31:0/119929696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc440106eb0 0x7fc44019e660 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:18.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.926+0000 7fc446ca9700 1 -- 172.21.15.31:0/119929696 >> 172.21.15.31:0/119929696 conn(0x7fc440102ab0 msgr2=0x7fc4401952d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:18.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.926+0000 7fc446ca9700 1 -- 172.21.15.31:0/119929696 shutdown_connections 2024-04-25T13:10:18.529 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:17.926+0000 7fc446ca9700 1 -- 172.21.15.31:0/119929696 wait complete. 2024-04-25T13:10:19.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:18 smithi031 ceph-mon[29457]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:19.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:18 smithi031 ceph-mon[29457]: Saving service mgr spec with placement count:2 2024-04-25T13:10:19.168 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:18 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.041+0000 7fbcce9b2700 1 Processor -- start 2024-04-25T13:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.041+0000 7fbcce9b2700 1 -- start start 2024-04-25T13:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.041+0000 7fbcce9b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc81078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.041+0000 7fbcce9b2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8107ec0 con 0x7fbcc81074d0 2024-04-25T13:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.041+0000 7fbcc7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc81078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.041+0000 7fbcc7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc81078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34122/0 (socket says 172.21.15.31:34122) 2024-04-25T13:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.041+0000 7fbcc7fff700 1 -- 172.21.15.31:0/4228833095 learned_addr learned my addr 172.21.15.31:0/4228833095 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.042+0000 7fbcc7fff700 1 -- 172.21.15.31:0/4228833095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc81086d0 con 0x7fbcc81074d0 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.042+0000 7fbcc7fff700 1 --2- 172.21.15.31:0/4228833095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc81078f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0098a0 tx=0x7fbcbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdc6d01d982b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.042+0000 7fbcc6ffd700 1 -- 172.21.15.31:0/4228833095 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcbc004160 con 0x7fbcc81074d0 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.042+0000 7fbcc6ffd700 1 -- 172.21.15.31:0/4228833095 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcbc004300 con 0x7fbcc81074d0 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.042+0000 7fbcc6ffd700 1 -- 172.21.15.31:0/4228833095 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcbc011a20 con 0x7fbcc81074d0 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.043+0000 7fbcce9b2700 1 -- 172.21.15.31:0/4228833095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 msgr2=0x7fbcc81078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.043+0000 7fbcce9b2700 1 --2- 172.21.15.31:0/4228833095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc81078f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0098a0 tx=0x7fbcbc009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.043+0000 7fbcce9b2700 1 -- 172.21.15.31:0/4228833095 shutdown_connections 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.043+0000 7fbcce9b2700 1 --2- 172.21.15.31:0/4228833095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc81078f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.043+0000 7fbcce9b2700 1 -- 172.21.15.31:0/4228833095 >> 172.21.15.31:0/4228833095 conn(0x7fbcc8102ad0 msgr2=0x7fbcc8104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.043+0000 7fbcce9b2700 1 -- 172.21.15.31:0/4228833095 shutdown_connections 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.043+0000 7fbcce9b2700 1 -- 172.21.15.31:0/4228833095 wait complete. 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.044+0000 7fbcce9b2700 1 Processor -- start 2024-04-25T13:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.044+0000 7fbcce9b2700 1 -- start start 2024-04-25T13:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.044+0000 7fbcce9b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.044+0000 7fbcce9b2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc810ba80 con 0x7fbcc81074d0 2024-04-25T13:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.044+0000 7fbcc7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.044+0000 7fbcc7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34138/0 (socket says 172.21.15.31:34138) 2024-04-25T13:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.044+0000 7fbcc7fff700 1 -- 172.21.15.31:0/2077169249 learned_addr learned my addr 172.21.15.31:0/2077169249 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.045+0000 7fbcc7fff700 1 -- 172.21.15.31:0/2077169249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc819c9d0 con 0x7fbcc81074d0 2024-04-25T13:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.045+0000 7fbcc7fff700 1 --2- 172.21.15.31:0/2077169249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc819c490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0116e0 tx=0x7fbcbc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.045+0000 7fbcc57fa700 1 -- 172.21.15.31:0/2077169249 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcbc0039e0 con 0x7fbcc81074d0 2024-04-25T13:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.045+0000 7fbcc57fa700 1 -- 172.21.15.31:0/2077169249 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcbc01b8f0 con 0x7fbcc81074d0 2024-04-25T13:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.045+0000 7fbcc57fa700 1 -- 172.21.15.31:0/2077169249 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcbc023760 con 0x7fbcc81074d0 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.045+0000 7fbcce9b2700 1 -- 172.21.15.31:0/2077169249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc819cc60 con 0x7fbcc81074d0 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.045+0000 7fbcce9b2700 1 -- 172.21.15.31:0/2077169249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc819f950 con 0x7fbcc81074d0 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.046+0000 7fbcc57fa700 1 -- 172.21.15.31:0/2077169249 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbcbc023940 con 0x7fbcc81074d0 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.046+0000 7fbcce9b2700 1 -- 172.21.15.31:0/2077169249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc810bc00 con 0x7fbcc81074d0 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.046+0000 7fbcc57fa700 1 --2- 172.21.15.31:0/2077169249 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fbcb00395b0 0x7fbcb003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.046+0000 7fbcc57fa700 1 -- 172.21.15.31:0/2077169249 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbcbc04e000 con 0x7fbcc81074d0 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.046+0000 7fbcc77fe700 1 --2- 172.21.15.31:0/2077169249 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fbcb00395b0 0x7fbcb003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.047+0000 7fbcc77fe700 1 --2- 172.21.15.31:0/2077169249 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fbcb00395b0 0x7fbcb003ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006fd0 tx=0x7fbcb8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.051+0000 7fbcc57fa700 1 -- 172.21.15.31:0/2077169249 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcbc01f030 con 0x7fbcc81074d0 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.261+0000 7fbcce9b2700 1 -- 172.21.15.31:0/2077169249 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fbcc802e250 con 0x7fbcc81074d0 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.264+0000 7fbcc57fa700 1 -- 172.21.15.31:0/2077169249 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fbcbc01adf0 con 0x7fbcc81074d0 2024-04-25T13:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.267+0000 7fbcce9b2700 1 -- 172.21.15.31:0/2077169249 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fbcb00395b0 msgr2=0x7fbcb003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.267+0000 7fbcce9b2700 1 --2- 172.21.15.31:0/2077169249 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fbcb00395b0 0x7fbcb003ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006fd0 tx=0x7fbcb8006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.267+0000 7fbcce9b2700 1 -- 172.21.15.31:0/2077169249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 msgr2=0x7fbcc819c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.267+0000 7fbcce9b2700 1 --2- 172.21.15.31:0/2077169249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc819c490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0116e0 tx=0x7fbcbc011aa0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.267+0000 7fbcce9b2700 1 -- 172.21.15.31:0/2077169249 shutdown_connections 2024-04-25T13:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.267+0000 7fbcce9b2700 1 --2- 172.21.15.31:0/2077169249 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7fbcb00395b0 0x7fbcb003ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.267+0000 7fbcce9b2700 1 --2- 172.21.15.31:0/2077169249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcc81074d0 0x7fbcc819c490 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.267+0000 7fbcce9b2700 1 -- 172.21.15.31:0/2077169249 >> 172.21.15.31:0/2077169249 conn(0x7fbcc8102ad0 msgr2=0x7fbcc8103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.267+0000 7fbcce9b2700 1 -- 172.21.15.31:0/2077169249 shutdown_connections 2024-04-25T13:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:20.267+0000 7fbcce9b2700 1 -- 172.21.15.31:0/2077169249 wait complete. 2024-04-25T13:10:21.121 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:20 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:21.121 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:20 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2077169249' entity='client.admin' 2024-04-25T13:10:21.121 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:20 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:22.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:22 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3057619197' entity='client.admin' 2024-04-25T13:10:22.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:22 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:22.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.226+0000 7f3f8dab4700 1 Processor -- start 2024-04-25T13:10:22.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.226+0000 7f3f8dab4700 1 -- start start 2024-04-25T13:10:22.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.226+0000 7f3f8dab4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:22.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.226+0000 7f3f8dab4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8807c8b0 con 0x7f3f8807de30 2024-04-25T13:10:22.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.226+0000 7f3f877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:22.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.226+0000 7f3f877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34152/0 (socket says 172.21.15.31:34152) 2024-04-25T13:10:22.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.226+0000 7f3f877fe700 1 -- 172.21.15.31:0/3145543182 learned_addr learned my addr 172.21.15.31:0/3145543182 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:22.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.227+0000 7f3f877fe700 1 -- 172.21.15.31:0/3145543182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8807ca30 con 0x7f3f8807de30 2024-04-25T13:10:22.680 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.227+0000 7f3f877fe700 1 --2- 172.21.15.31:0/3145543182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8807c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f780098a0 tx=0x7f3f78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d394793225e266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.227+0000 7f3f867fc700 1 -- 172.21.15.31:0/3145543182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f78004160 con 0x7f3f8807de30 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.227+0000 7f3f867fc700 1 -- 172.21.15.31:0/3145543182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f78004300 con 0x7f3f8807de30 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.227+0000 7f3f867fc700 1 -- 172.21.15.31:0/3145543182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f78011a20 con 0x7f3f8807de30 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.228+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3145543182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 msgr2=0x7f3f8807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.228+0000 7f3f8dab4700 1 --2- 172.21.15.31:0/3145543182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8807c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f780098a0 tx=0x7f3f78009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.228+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3145543182 shutdown_connections 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.228+0000 7f3f8dab4700 1 --2- 172.21.15.31:0/3145543182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8807c2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.228+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3145543182 >> 172.21.15.31:0/3145543182 conn(0x7f3f88103bc0 msgr2=0x7f3f88105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.228+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3145543182 shutdown_connections 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.228+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3145543182 wait complete. 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.228+0000 7f3f8dab4700 1 Processor -- start 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.228+0000 7f3f8dab4700 1 -- start start 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f8dab4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f8dab4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8810eba0 con 0x7f3f8807de30 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34156/0 (socket says 172.21.15.31:34156) 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f877fe700 1 -- 172.21.15.31:0/3057619197 learned_addr learned my addr 172.21.15.31:0/3057619197 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f877fe700 1 -- 172.21.15.31:0/3057619197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8819eb10 con 0x7f3f8807de30 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f877fe700 1 --2- 172.21.15.31:0/3057619197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8819e5d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f780116e0 tx=0x7f3f78011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f84ff9700 1 -- 172.21.15.31:0/3057619197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f78003a60 con 0x7f3f8807de30 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f84ff9700 1 -- 172.21.15.31:0/3057619197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7801b8f0 con 0x7f3f8807de30 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f84ff9700 1 -- 172.21.15.31:0/3057619197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f78019860 con 0x7f3f8807de30 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3057619197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8819eda0 con 0x7f3f8807de30 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.229+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3057619197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f881a1a90 con 0x7f3f8807de30 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.230+0000 7f3f84ff9700 1 -- 172.21.15.31:0/3057619197 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3f7801b440 con 0x7f3f8807de30 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.230+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3057619197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f880640a0 con 0x7f3f8807de30 2024-04-25T13:10:22.682 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.230+0000 7f3f84ff9700 1 --2- 172.21.15.31:0/3057619197 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f3f700395b0 0x7f3f7003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:22.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.230+0000 7f3f84ff9700 1 -- 172.21.15.31:0/3057619197 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3f7804dc90 con 0x7f3f8807de30 2024-04-25T13:10:22.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.231+0000 7f3f86ffd700 1 --2- 172.21.15.31:0/3057619197 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f3f700395b0 0x7f3f7003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:22.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.231+0000 7f3f86ffd700 1 --2- 172.21.15.31:0/3057619197 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f3f700395b0 0x7f3f7003ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006fd0 tx=0x7f3f7c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:22.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.235+0000 7f3f84ff9700 1 -- 172.21.15.31:0/3057619197 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f7801f030 con 0x7f3f8807de30 2024-04-25T13:10:22.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.455+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3057619197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f3f881a1c60 con 0x7f3f8807de30 2024-04-25T13:10:22.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.458+0000 7f3f84ff9700 1 -- 172.21.15.31:0/3057619197 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f3f780223b0 con 0x7f3f8807de30 2024-04-25T13:10:22.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.460+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3057619197 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f3f700395b0 msgr2=0x7f3f7003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:22.683 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.460+0000 7f3f8dab4700 1 --2- 172.21.15.31:0/3057619197 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f3f700395b0 0x7f3f7003ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006fd0 tx=0x7f3f7c006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:22.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.460+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3057619197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 msgr2=0x7f3f8819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:22.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.460+0000 7f3f8dab4700 1 --2- 172.21.15.31:0/3057619197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8819e5d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f780116e0 tx=0x7f3f78011aa0 comp rx=0 tx=0).stop 2024-04-25T13:10:22.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.461+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3057619197 shutdown_connections 2024-04-25T13:10:22.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.461+0000 7f3f8dab4700 1 --2- 172.21.15.31:0/3057619197 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f3f700395b0 0x7f3f7003ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:22.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.461+0000 7f3f8dab4700 1 --2- 172.21.15.31:0/3057619197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f8807de30 0x7f3f8819e5d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:22.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.461+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3057619197 >> 172.21.15.31:0/3057619197 conn(0x7f3f88103bc0 msgr2=0x7f3f88104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:22.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.461+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3057619197 shutdown_connections 2024-04-25T13:10:22.684 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:21.461+0000 7f3f8dab4700 1 -- 172.21.15.31:0/3057619197 wait complete. 2024-04-25T13:10:22.684 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2024-04-25T13:10:24.580 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:24 smithi031 ceph-mon[29457]: from='mgr.14116 172.21.15.31:0/3147738812' entity='mgr.x' 2024-04-25T13:10:24.580 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:24 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2250356198' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T13:10:24.580 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:24 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: ignoring --setuser ceph since I am not root 2024-04-25T13:10:24.580 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:24 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: ignoring --setgroup ceph since I am not root 2024-04-25T13:10:24.580 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:24 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:24.580+0000 7f7a1422b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:10:24.885 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:24 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:24.777+0000 7f7a1422b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:10:24.885 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:24 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:24.885+0000 7f7a1422b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:10:25.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.129+0000 7f1a6d67c700 1 Processor -- start 2024-04-25T13:10:25.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.129+0000 7f1a6d67c700 1 -- start start 2024-04-25T13:10:25.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.129+0000 7f1a6d67c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a6810b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:25.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.129+0000 7f1a6d67c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6807be50 con 0x7f1a68109050 2024-04-25T13:10:25.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.129+0000 7f1a66ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a6810b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:25.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.129+0000 7f1a66ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a6810b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39658/0 (socket says 172.21.15.31:39658) 2024-04-25T13:10:25.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.129+0000 7f1a66ffd700 1 -- 172.21.15.31:0/3190091132 learned_addr learned my addr 172.21.15.31:0/3190091132 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:25.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.130+0000 7f1a66ffd700 1 -- 172.21.15.31:0/3190091132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6810b980 con 0x7f1a68109050 2024-04-25T13:10:25.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.130+0000 7f1a66ffd700 1 --2- 172.21.15.31:0/3190091132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a6810b440 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0098a0 tx=0x7f1a5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cad70d18ade3020 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:25.071 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.130+0000 7f1a65ffb700 1 -- 172.21.15.31:0/3190091132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a5c004160 con 0x7f1a68109050 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.130+0000 7f1a65ffb700 1 -- 172.21.15.31:0/3190091132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a5c004300 con 0x7f1a68109050 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.130+0000 7f1a65ffb700 1 -- 172.21.15.31:0/3190091132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a5c011a20 con 0x7f1a68109050 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.131+0000 7f1a6d67c700 1 -- 172.21.15.31:0/3190091132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 msgr2=0x7f1a6810b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.131+0000 7f1a6d67c700 1 --2- 172.21.15.31:0/3190091132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a6810b440 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0098a0 tx=0x7f1a5c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.131+0000 7f1a6d67c700 1 -- 172.21.15.31:0/3190091132 shutdown_connections 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.131+0000 7f1a6d67c700 1 --2- 172.21.15.31:0/3190091132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a6810b440 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.131+0000 7f1a6d67c700 1 -- 172.21.15.31:0/3190091132 >> 172.21.15.31:0/3190091132 conn(0x7f1a68102a60 msgr2=0x7f1a68104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.131+0000 7f1a6d67c700 1 -- 172.21.15.31:0/3190091132 shutdown_connections 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.131+0000 7f1a6d67c700 1 -- 172.21.15.31:0/3190091132 wait complete. 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.132+0000 7f1a6d67c700 1 Processor -- start 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.132+0000 7f1a6d67c700 1 -- start start 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.132+0000 7f1a6d67c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a681088d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.132+0000 7f1a6d67c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6810c980 con 0x7f1a68109050 2024-04-25T13:10:25.072 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.132+0000 7f1a66ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a681088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.132+0000 7f1a66ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a681088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39668/0 (socket says 172.21.15.31:39668) 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.132+0000 7f1a66ffd700 1 -- 172.21.15.31:0/2250356198 learned_addr learned my addr 172.21.15.31:0/2250356198 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.133+0000 7f1a66ffd700 1 -- 172.21.15.31:0/2250356198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a68108e10 con 0x7f1a68109050 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.133+0000 7f1a66ffd700 1 --2- 172.21.15.31:0/2250356198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a681088d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c000c00 tx=0x7f1a5c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.133+0000 7f1a4ffff700 1 -- 172.21.15.31:0/2250356198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a5c003aa0 con 0x7f1a68109050 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.133+0000 7f1a4ffff700 1 -- 172.21.15.31:0/2250356198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a5c01b8f0 con 0x7f1a68109050 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.133+0000 7f1a6d67c700 1 -- 172.21.15.31:0/2250356198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a68107000 con 0x7f1a68109050 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.133+0000 7f1a4ffff700 1 -- 172.21.15.31:0/2250356198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a5c023870 con 0x7f1a68109050 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.133+0000 7f1a6d67c700 1 -- 172.21.15.31:0/2250356198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a68107540 con 0x7f1a68109050 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.134+0000 7f1a6d67c700 1 -- 172.21.15.31:0/2250356198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a680640a0 con 0x7f1a68109050 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.134+0000 7f1a4ffff700 1 -- 172.21.15.31:0/2250356198 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1a5c023b70 con 0x7f1a68109050 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.134+0000 7f1a4ffff700 1 --2- 172.21.15.31:0/2250356198 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f1a500395b0 0x7f1a5003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.134+0000 7f1a4ffff700 1 -- 172.21.15.31:0/2250356198 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1a5c04de80 con 0x7f1a68109050 2024-04-25T13:10:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.135+0000 7f1a667fc700 1 --2- 172.21.15.31:0/2250356198 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f1a500395b0 0x7f1a5003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.135+0000 7f1a667fc700 1 --2- 172.21.15.31:0/2250356198 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f1a500395b0 0x7f1a5003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a58006fd0 tx=0x7f1a58006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.139+0000 7f1a4ffff700 1 -- 172.21.15.31:0/2250356198 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a5c01f030 con 0x7f1a68109050 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:23.411+0000 7f1a6d67c700 1 -- 172.21.15.31:0/2250356198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f1a68107e20 con 0x7f1a68109050 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.395+0000 7f1a4ffff700 1 -- 172.21.15.31:0/2250356198 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f1a5c016260 con 0x7f1a68109050 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.398+0000 7f1a4ffff700 1 -- 172.21.15.31:0/2250356198 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f1a5c02c720 con 0x7f1a68109050 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.399+0000 7f1a6d67c700 1 -- 172.21.15.31:0/2250356198 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f1a500395b0 msgr2=0x7f1a5003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.399+0000 7f1a6d67c700 1 --2- 172.21.15.31:0/2250356198 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f1a500395b0 0x7f1a5003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a58006fd0 tx=0x7f1a58006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.399+0000 7f1a6d67c700 1 -- 172.21.15.31:0/2250356198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 msgr2=0x7f1a681088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.399+0000 7f1a6d67c700 1 --2- 172.21.15.31:0/2250356198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a681088d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c000c00 tx=0x7f1a5c011aa0 comp rx=0 tx=0).stop 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.400+0000 7f1a6d67c700 1 -- 172.21.15.31:0/2250356198 shutdown_connections 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.400+0000 7f1a6d67c700 1 --2- 172.21.15.31:0/2250356198 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f1a500395b0 0x7f1a5003ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.400+0000 7f1a6d67c700 1 --2- 172.21.15.31:0/2250356198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a68109050 0x7f1a681088d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:25.074 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.400+0000 7f1a6d67c700 1 -- 172.21.15.31:0/2250356198 >> 172.21.15.31:0/2250356198 conn(0x7f1a68102a60 msgr2=0x7f1a68103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:25.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.400+0000 7f1a6d67c700 1 -- 172.21.15.31:0/2250356198 shutdown_connections 2024-04-25T13:10:25.075 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:24.400+0000 7f1a6d67c700 1 -- 172.21.15.31:0/2250356198 wait complete. 2024-04-25T13:10:25.415 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:25 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2250356198' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T13:10:25.415 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:25 smithi031 ceph-mon[29457]: mgrmap e9: x(active, since 20s) 2024-04-25T13:10:25.415 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:25 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:25.218+0000 7f7a1422b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:10:25.416 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:25 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:25.307+0000 7f7a1422b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:10:26.721 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:26 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1469709831' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T13:10:26.721 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:26 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:26.619+0000 7f7a1422b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:10:26.732 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:10:26.732 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T13:10:26.732 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.832+0000 7f634a7b9700 1 Processor -- start 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.832+0000 7f634a7b9700 1 -- start start 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.832+0000 7f634a7b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634410b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.832+0000 7f634a7b9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f634407be90 con 0x7f6344109040 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.833+0000 7f6343fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634410b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.833+0000 7f6343fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634410b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39694/0 (socket says 172.21.15.31:39694) 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.833+0000 7f6343fff700 1 -- 172.21.15.31:0/4152834514 learned_addr learned my addr 172.21.15.31:0/4152834514 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.833+0000 7f6343fff700 1 -- 172.21.15.31:0/4152834514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634410b970 con 0x7f6344109040 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.833+0000 7f6343fff700 1 --2- 172.21.15.31:0/4152834514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634410b430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63340098a0 tx=0x7f6334009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e6d30a12b7c911 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:26.733 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.833+0000 7f6342ffd700 1 -- 172.21.15.31:0/4152834514 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6334004160 con 0x7f6344109040 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.833+0000 7f6342ffd700 1 -- 172.21.15.31:0/4152834514 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6334004300 con 0x7f6344109040 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.833+0000 7f6342ffd700 1 -- 172.21.15.31:0/4152834514 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6334011a20 con 0x7f6344109040 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.834+0000 7f634a7b9700 1 -- 172.21.15.31:0/4152834514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 msgr2=0x7f634410b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.834+0000 7f634a7b9700 1 --2- 172.21.15.31:0/4152834514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634410b430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63340098a0 tx=0x7f6334009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.834+0000 7f634a7b9700 1 -- 172.21.15.31:0/4152834514 shutdown_connections 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.834+0000 7f634a7b9700 1 --2- 172.21.15.31:0/4152834514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634410b430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.834+0000 7f634a7b9700 1 -- 172.21.15.31:0/4152834514 >> 172.21.15.31:0/4152834514 conn(0x7f6344102ad0 msgr2=0x7f6344104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.834+0000 7f634a7b9700 1 -- 172.21.15.31:0/4152834514 shutdown_connections 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.834+0000 7f634a7b9700 1 -- 172.21.15.31:0/4152834514 wait complete. 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.835+0000 7f634a7b9700 1 Processor -- start 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.835+0000 7f634a7b9700 1 -- start start 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.835+0000 7f634a7b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.835+0000 7f634a7b9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f634410c980 con 0x7f6344109040 2024-04-25T13:10:26.734 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.835+0000 7f6343fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.835+0000 7f6343fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39706/0 (socket says 172.21.15.31:39706) 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.835+0000 7f6343fff700 1 -- 172.21.15.31:0/1469709831 learned_addr learned my addr 172.21.15.31:0/1469709831 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.836+0000 7f6343fff700 1 -- 172.21.15.31:0/1469709831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634419c970 con 0x7f6344109040 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.836+0000 7f6343fff700 1 --2- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634419c430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f6334011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.836+0000 7f63417fa700 1 -- 172.21.15.31:0/1469709831 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6334003aa0 con 0x7f6344109040 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.836+0000 7f63417fa700 1 -- 172.21.15.31:0/1469709831 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f633401b8f0 con 0x7f6344109040 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.836+0000 7f634a7b9700 1 -- 172.21.15.31:0/1469709831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634419cc00 con 0x7f6344109040 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.836+0000 7f63417fa700 1 -- 172.21.15.31:0/1469709831 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6334023870 con 0x7f6344109040 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.836+0000 7f634a7b9700 1 -- 172.21.15.31:0/1469709831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634419f8f0 con 0x7f6344109040 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.837+0000 7f634a7b9700 1 -- 172.21.15.31:0/1469709831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63440640a0 con 0x7f6344109040 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.837+0000 7f63417fa700 1 -- 172.21.15.31:0/1469709831 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f6334023b20 con 0x7f6344109040 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.838+0000 7f63417fa700 1 --2- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f632c039660 0x7f632c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.838+0000 7f63417fa700 1 -- 172.21.15.31:0/1469709831 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f633404de60 con 0x7f6344109040 2024-04-25T13:10:26.735 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.838+0000 7f63437fe700 1 -- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f632c039660 msgr2=0x7f632c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3047311576 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.838+0000 7f63437fe700 1 --2- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f632c039660 0x7f632c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:25.843+0000 7f63417fa700 1 -- 172.21.15.31:0/1469709831 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f633401f030 con 0x7f6344109040 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.039+0000 7f63437fe700 1 -- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f632c039660 msgr2=0x7f632c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3047311576 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.039+0000 7f63437fe700 1 --2- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f632c039660 0x7f632c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.111+0000 7f634a7b9700 1 -- 172.21.15.31:0/1469709831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6344107300 con 0x7f6344109040 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.112+0000 7f63417fa700 1 -- 172.21.15.31:0/1469709831 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f6334016260 con 0x7f6344109040 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.114+0000 7f634a7b9700 1 -- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f632c039660 msgr2=0x7f632c03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.114+0000 7f634a7b9700 1 --2- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f632c039660 0x7f632c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.114+0000 7f634a7b9700 1 -- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 msgr2=0x7f634419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.114+0000 7f634a7b9700 1 --2- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634419c430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f6334011aa0 comp rx=0 tx=0).stop 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.114+0000 7f634a7b9700 1 -- 172.21.15.31:0/1469709831 shutdown_connections 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.114+0000 7f634a7b9700 1 --2- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f632c039660 0x7f632c03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.114+0000 7f634a7b9700 1 --2- 172.21.15.31:0/1469709831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6344109040 0x7f634419c430 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:26.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.114+0000 7f634a7b9700 1 -- 172.21.15.31:0/1469709831 >> 172.21.15.31:0/1469709831 conn(0x7f6344102ad0 msgr2=0x7f63441035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:26.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.114+0000 7f634a7b9700 1 -- 172.21.15.31:0/1469709831 shutdown_connections 2024-04-25T13:10:26.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:26.114+0000 7f634a7b9700 1 -- 172.21.15.31:0/1469709831 wait complete. 2024-04-25T13:10:26.737 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-04-25T13:10:26.737 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 9... 2024-04-25T13:10:26.975 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:26 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:26.811+0000 7f7a1422b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:10:27.233 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:26 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:26.998+0000 7f7a1422b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:10:27.636 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:27 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:27.315+0000 7f7a1422b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:10:27.637 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:27 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:27.414+0000 7f7a1422b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:10:27.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:27 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:27.637+0000 7f7a1422b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:10:27.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:27 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:27.841+0000 7f7a1422b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:10:28.733 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:28.476+0000 7f7a1422b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:10:29.233 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:28.923+0000 7f7a1422b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:10:29.233 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:29 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:29.025+0000 7f7a1422b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:10:29.733 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:29 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:29.240+0000 7f7a1422b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:10:30.148 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:29 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:29.794+0000 7f7a1422b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:10:30.148 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:29 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:29.892+0000 7f7a1422b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:10:30.483 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:30.149+0000 7f7a1422b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:10:30.483 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:30.290+0000 7f7a1422b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:10:30.483 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:30.391+0000 7f7a1422b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:10:30.828 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:30.513+0000 7f7a1422b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:10:30.828 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:30.828+0000 7f7a1422b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:10:31.733 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:31.472+0000 7f7a1422b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:10:31.733 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:10:31.700+0000 7f7a1422b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:10:32.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-mon[29457]: Active manager daemon x restarted 2024-04-25T13:10:32.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-mon[29457]: Activating manager daemon x 2024-04-25T13:10:32.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-mon[29457]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T13:10:32.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-mon[29457]: mgrmap e10: x(active, starting, since 0.00343643s) 2024-04-25T13:10:32.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:10:32.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:10:32.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:10:32.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:10:32.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:10:32.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-mon[29457]: Manager daemon x is now available 2024-04-25T13:10:32.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:32.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:10:32.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:10:32.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:32 smithi031 ceph-mon[29457]: mgrmap e11: x(active, since 1.00618s) 2024-04-25T13:10:33.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:10:33.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T13:10:33.473 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.171+0000 7f9e079a0700 1 Processor -- start 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.171+0000 7f9e079a0700 1 -- start start 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.172+0000 7f9e079a0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e001072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.172+0000 7f9e079a0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9e00063ce0 con 0x7f9e00106eb0 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.172+0000 7f9e0573c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e001072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.172+0000 7f9e0573c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e001072d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39720/0 (socket says 172.21.15.31:39720) 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.172+0000 7f9e0573c700 1 -- 172.21.15.31:0/14733666 learned_addr learned my addr 172.21.15.31:0/14733666 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.172+0000 7f9e0573c700 1 -- 172.21.15.31:0/14733666 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e00107810 con 0x7f9e00106eb0 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.173+0000 7f9e0573c700 1 --2- 172.21.15.31:0/14733666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e001072d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0098a0 tx=0x7f9dfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ad4c9fde4f62ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.173+0000 7f9df7fff700 1 -- 172.21.15.31:0/14733666 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dfc004160 con 0x7f9e00106eb0 2024-04-25T13:10:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.173+0000 7f9df7fff700 1 -- 172.21.15.31:0/14733666 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dfc004300 con 0x7f9e00106eb0 2024-04-25T13:10:33.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.173+0000 7f9df7fff700 1 -- 172.21.15.31:0/14733666 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dfc011a20 con 0x7f9e00106eb0 2024-04-25T13:10:33.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.173+0000 7f9e079a0700 1 -- 172.21.15.31:0/14733666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 msgr2=0x7f9e001072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:33.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.173+0000 7f9e079a0700 1 --2- 172.21.15.31:0/14733666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e001072d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0098a0 tx=0x7f9dfc009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:33.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.174+0000 7f9e079a0700 1 -- 172.21.15.31:0/14733666 shutdown_connections 2024-04-25T13:10:33.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.174+0000 7f9e079a0700 1 --2- 172.21.15.31:0/14733666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e001072d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:33.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.174+0000 7f9e079a0700 1 -- 172.21.15.31:0/14733666 >> 172.21.15.31:0/14733666 conn(0x7f9e00102ab0 msgr2=0x7f9e00104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:33.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.174+0000 7f9e079a0700 1 -- 172.21.15.31:0/14733666 shutdown_connections 2024-04-25T13:10:33.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.174+0000 7f9e079a0700 1 -- 172.21.15.31:0/14733666 wait complete. 2024-04-25T13:10:33.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.174+0000 7f9e079a0700 1 Processor -- start 2024-04-25T13:10:33.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.174+0000 7f9e079a0700 1 -- start start 2024-04-25T13:10:33.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.174+0000 7f9e079a0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e0019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.174+0000 7f9e079a0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9e00108880 con 0x7f9e00106eb0 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.175+0000 7f9e0573c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e0019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.175+0000 7f9e0573c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e0019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39728/0 (socket says 172.21.15.31:39728) 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.175+0000 7f9e0573c700 1 -- 172.21.15.31:0/4011365072 learned_addr learned my addr 172.21.15.31:0/4011365072 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.175+0000 7f9e0573c700 1 -- 172.21.15.31:0/4011365072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0019eb50 con 0x7f9e00106eb0 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.175+0000 7f9e0573c700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e0019e610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc000c00 tx=0x7f9dfc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.175+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dfc003aa0 con 0x7f9e00106eb0 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.175+0000 7f9e079a0700 1 -- 172.21.15.31:0/4011365072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0019ede0 con 0x7f9e00106eb0 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.175+0000 7f9e079a0700 1 -- 172.21.15.31:0/4011365072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e001a1ad0 con 0x7f9e00106eb0 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.176+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dfc01b8f0 con 0x7f9e00106eb0 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.176+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dfc023870 con 0x7f9e00106eb0 2024-04-25T13:10:33.476 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.176+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f9dfc023b20 con 0x7f9e00106eb0 2024-04-25T13:10:33.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.177+0000 7f9df67fc700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 0x7f9dec03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:33.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.177+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 --> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9dec03c230 con 0x7f9dec039660 2024-04-25T13:10:33.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.177+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9dfc04de30 con 0x7f9e00106eb0 2024-04-25T13:10:33.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.177+0000 7f9e04f3b700 1 -- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 msgr2=0x7f9dec03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3047311576 2024-04-25T13:10:33.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.177+0000 7f9e04f3b700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 0x7f9dec03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:10:33.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.378+0000 7f9e04f3b700 1 -- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 msgr2=0x7f9dec03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3047311576 2024-04-25T13:10:33.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.378+0000 7f9e04f3b700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 0x7f9dec03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:10:33.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.779+0000 7f9e04f3b700 1 -- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 msgr2=0x7f9dec03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3047311576 2024-04-25T13:10:33.477 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:27.779+0000 7f9e04f3b700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 0x7f9dec03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:10:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:28.580+0000 7f9e04f3b700 1 -- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 msgr2=0x7f9dec03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3047311576 2024-04-25T13:10:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:28.580+0000 7f9e04f3b700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 0x7f9dec03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:10:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:30.181+0000 7f9e04f3b700 1 -- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 msgr2=0x7f9dec03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3047311576 2024-04-25T13:10:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:30.181+0000 7f9e04f3b700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 0x7f9dec03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T13:10:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:31.704+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f9dfc02c720 con 0x7f9e00106eb0 2024-04-25T13:10:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:31.704+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 msgr2=0x7f9dec03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:10:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:31.704+0000 7f9df67fc700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 0x7f9dec03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.706+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f9dfc00bce0 con 0x7f9e00106eb0 2024-04-25T13:10:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.706+0000 7f9df67fc700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dec03cff0 0x7f9dec03f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.706+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9dec03c230 con 0x7f9dec03cff0 2024-04-25T13:10:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.706+0000 7f9e04f3b700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dec03cff0 0x7f9dec03f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:33.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.707+0000 7f9e04f3b700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dec03cff0 0x7f9dec03f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9df0003a10 tx=0x7f9df0009270 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:33.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.708+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f9dec03c230 con 0x7f9dec03cff0 2024-04-25T13:10:33.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.714+0000 7f9e079a0700 1 -- 172.21.15.31:0/4011365072 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9de4002630 con 0x7f9dec03cff0 2024-04-25T13:10:33.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9df67fc700 1 -- 172.21.15.31:0/4011365072 <== mgr.14148 v2:172.21.15.31:6800/958678691 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f9de4002630 con 0x7f9dec03cff0 2024-04-25T13:10:33.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9e079a0700 1 -- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dec03cff0 msgr2=0x7f9dec03f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:33.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9e079a0700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dec03cff0 0x7f9dec03f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9df0003a10 tx=0x7f9df0009270 comp rx=0 tx=0).stop 2024-04-25T13:10:33.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9e079a0700 1 -- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 msgr2=0x7f9e0019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:33.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9e079a0700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e0019e610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc000c00 tx=0x7f9dfc011aa0 comp rx=0 tx=0).stop 2024-04-25T13:10:33.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9e079a0700 1 -- 172.21.15.31:0/4011365072 shutdown_connections 2024-04-25T13:10:33.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9e079a0700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00106eb0 0x7f9e0019e610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:33.479 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9e079a0700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dec03cff0 0x7f9dec03f3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:33.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9e079a0700 1 --2- 172.21.15.31:0/4011365072 >> [v2:172.21.15.31:6800/3047311576,v1:172.21.15.31:6801/3047311576] conn(0x7f9dec039660 0x7f9dec03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:33.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9e079a0700 1 -- 172.21.15.31:0/4011365072 >> 172.21.15.31:0/4011365072 conn(0x7f9e00102ab0 msgr2=0x7f9e001952d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:33.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9e079a0700 1 -- 172.21.15.31:0/4011365072 shutdown_connections 2024-04-25T13:10:33.480 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:32.715+0000 7f9e079a0700 1 -- 172.21.15.31:0/4011365072 wait complete. 2024-04-25T13:10:33.480 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 9 is available 2024-04-25T13:10:33.480 INFO:teuthology.orchestra.run.smithi031.stdout:Generating a dashboard self-signed certificate... 2024-04-25T13:10:33.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:33 smithi031 ceph-mon[29457]: [25/Apr/2024:13:10:33] ENGINE Bus STARTING 2024-04-25T13:10:34.871 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:34 smithi031 ceph-mon[29457]: [25/Apr/2024:13:10:33] ENGINE Serving on https://172.21.15.31:7150 2024-04-25T13:10:34.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:34 smithi031 ceph-mon[29457]: [25/Apr/2024:13:10:33] ENGINE Serving on http://172.21.15.31:8765 2024-04-25T13:10:34.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:34 smithi031 ceph-mon[29457]: [25/Apr/2024:13:10:33] ENGINE Bus STARTED 2024-04-25T13:10:34.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:34 smithi031 ceph-mon[29457]: mgrmap e12: x(active, since 2s) 2024-04-25T13:10:34.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:34 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:34.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:34 smithi031 ceph-mon[29457]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:34.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:34 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:34.872 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:34 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:34.880 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T13:10:34.880 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.912+0000 7fb50e333700 1 Processor -- start 2024-04-25T13:10:34.880 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.912+0000 7fb50e333700 1 -- start start 2024-04-25T13:10:34.880 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.912+0000 7fb50e333700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb5081078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:34.880 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.912+0000 7fb50e333700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb508107e90 con 0x7fb5081074a0 2024-04-25T13:10:34.880 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.913+0000 7fb507fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb5081078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:34.880 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.913+0000 7fb507fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb5081078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59094/0 (socket says 172.21.15.31:59094) 2024-04-25T13:10:34.880 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.913+0000 7fb507fff700 1 -- 172.21.15.31:0/2857269648 learned_addr learned my addr 172.21.15.31:0/2857269648 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.913+0000 7fb507fff700 1 -- 172.21.15.31:0/2857269648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5081086a0 con 0x7fb5081074a0 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.913+0000 7fb507fff700 1 --2- 172.21.15.31:0/2857269648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb5081078c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80098a0 tx=0x7fb4f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab7d9e4eda8ee18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.913+0000 7fb506ffd700 1 -- 172.21.15.31:0/2857269648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f8004160 con 0x7fb5081074a0 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.913+0000 7fb506ffd700 1 -- 172.21.15.31:0/2857269648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f8004300 con 0x7fb5081074a0 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.913+0000 7fb506ffd700 1 -- 172.21.15.31:0/2857269648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f8011a20 con 0x7fb5081074a0 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.914+0000 7fb50e333700 1 -- 172.21.15.31:0/2857269648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 msgr2=0x7fb5081078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.914+0000 7fb50e333700 1 --2- 172.21.15.31:0/2857269648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb5081078c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80098a0 tx=0x7fb4f8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.914+0000 7fb50e333700 1 -- 172.21.15.31:0/2857269648 shutdown_connections 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.914+0000 7fb50e333700 1 --2- 172.21.15.31:0/2857269648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb5081078c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.914+0000 7fb50e333700 1 -- 172.21.15.31:0/2857269648 >> 172.21.15.31:0/2857269648 conn(0x7fb508102a60 msgr2=0x7fb508104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.914+0000 7fb50e333700 1 -- 172.21.15.31:0/2857269648 shutdown_connections 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.914+0000 7fb50e333700 1 -- 172.21.15.31:0/2857269648 wait complete. 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.914+0000 7fb50e333700 1 Processor -- start 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.914+0000 7fb50e333700 1 -- start start 2024-04-25T13:10:34.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.915+0000 7fb50e333700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb50819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.915+0000 7fb50e333700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb50807afb0 con 0x7fb5081074a0 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.915+0000 7fb507fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb50819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.915+0000 7fb507fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb50819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59108/0 (socket says 172.21.15.31:59108) 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.915+0000 7fb507fff700 1 -- 172.21.15.31:0/3668574697 learned_addr learned my addr 172.21.15.31:0/3668574697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.915+0000 7fb507fff700 1 -- 172.21.15.31:0/3668574697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50819eb70 con 0x7fb5081074a0 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.915+0000 7fb507fff700 1 --2- 172.21.15.31:0/3668574697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb50819e630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009520 tx=0x7fb4f8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.915+0000 7fb5057fa700 1 -- 172.21.15.31:0/3668574697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f80039e0 con 0x7fb5081074a0 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.915+0000 7fb50e333700 1 -- 172.21.15.31:0/3668574697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50819ee00 con 0x7fb5081074a0 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.915+0000 7fb5057fa700 1 -- 172.21.15.31:0/3668574697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f801b8f0 con 0x7fb5081074a0 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.915+0000 7fb5057fa700 1 -- 172.21.15.31:0/3668574697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f8019770 con 0x7fb5081074a0 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.916+0000 7fb50e333700 1 -- 172.21.15.31:0/3668574697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5081a1af0 con 0x7fb5081074a0 2024-04-25T13:10:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.916+0000 7fb5057fa700 1 -- 172.21.15.31:0/3668574697 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fb4f8019a70 con 0x7fb5081074a0 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.916+0000 7fb50e333700 1 -- 172.21.15.31:0/3668574697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5080640a0 con 0x7fb5081074a0 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.917+0000 7fb5057fa700 1 --2- 172.21.15.31:0/3668574697 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4f00395f0 0x7fb4f003bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.917+0000 7fb5057fa700 1 -- 172.21.15.31:0/3668574697 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb4f804e1d0 con 0x7fb5081074a0 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.917+0000 7fb5077fe700 1 --2- 172.21.15.31:0/3668574697 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4f00395f0 0x7fb4f003bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.917+0000 7fb5077fe700 1 --2- 172.21.15.31:0/3668574697 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4f00395f0 0x7fb4f003bab0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc006fd0 tx=0x7fb4fc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:33.922+0000 7fb5057fa700 1 -- 172.21.15.31:0/3668574697 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4f801f030 con 0x7fb5081074a0 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.160+0000 7fb50e333700 1 -- 172.21.15.31:0/3668574697 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb5080753f0 con 0x7fb4f00395f0 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.306+0000 7fb5057fa700 1 -- 172.21.15.31:0/3668574697 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb5080753f0 con 0x7fb4f00395f0 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.309+0000 7fb50e333700 1 -- 172.21.15.31:0/3668574697 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4f00395f0 msgr2=0x7fb4f003bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.309+0000 7fb50e333700 1 --2- 172.21.15.31:0/3668574697 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4f00395f0 0x7fb4f003bab0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc006fd0 tx=0x7fb4fc006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.309+0000 7fb50e333700 1 -- 172.21.15.31:0/3668574697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 msgr2=0x7fb50819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.309+0000 7fb50e333700 1 --2- 172.21.15.31:0/3668574697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb50819e630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009520 tx=0x7fb4f8011aa0 comp rx=0 tx=0).stop 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.310+0000 7fb50e333700 1 -- 172.21.15.31:0/3668574697 shutdown_connections 2024-04-25T13:10:34.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.310+0000 7fb50e333700 1 --2- 172.21.15.31:0/3668574697 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4f00395f0 0x7fb4f003bab0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:34.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.310+0000 7fb50e333700 1 --2- 172.21.15.31:0/3668574697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5081074a0 0x7fb50819e630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:34.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.310+0000 7fb50e333700 1 -- 172.21.15.31:0/3668574697 >> 172.21.15.31:0/3668574697 conn(0x7fb508102a60 msgr2=0x7fb508190860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:34.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.310+0000 7fb50e333700 1 -- 172.21.15.31:0/3668574697 shutdown_connections 2024-04-25T13:10:34.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:34.310+0000 7fb50e333700 1 -- 172.21.15.31:0/3668574697 wait complete. 2024-04-25T13:10:34.884 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2024-04-25T13:10:36.395 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:36.395 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:37.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$PPW31lrcBdWxe1dKkMW6DO42oodYeFYsgAsirF5Gm4BXdri0E0Yfe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714050635, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T13:10:37.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.328+0000 7f4ea3b27700 1 Processor -- start 2024-04-25T13:10:37.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.328+0000 7f4ea3b27700 1 -- start start 2024-04-25T13:10:37.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.328+0000 7f4ea3b27700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c10b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:37.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.328+0000 7f4ea3b27700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c07be90 con 0x7f4e9c109040 2024-04-25T13:10:37.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.328+0000 7f4ea18c3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c10b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:37.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.328+0000 7f4ea18c3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c10b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59112/0 (socket says 172.21.15.31:59112) 2024-04-25T13:10:37.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.328+0000 7f4ea18c3700 1 -- 172.21.15.31:0/2739677948 learned_addr learned my addr 172.21.15.31:0/2739677948 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:37.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.329+0000 7f4ea18c3700 1 -- 172.21.15.31:0/2739677948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c10b970 con 0x7f4e9c109040 2024-04-25T13:10:37.393 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.329+0000 7f4ea18c3700 1 --2- 172.21.15.31:0/2739677948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c10b430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4e980098a0 tx=0x7f4e98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8754b48176dcc84b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.329+0000 7f4ea08c1700 1 -- 172.21.15.31:0/2739677948 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e98004160 con 0x7f4e9c109040 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.329+0000 7f4ea08c1700 1 -- 172.21.15.31:0/2739677948 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e98004300 con 0x7f4e9c109040 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.329+0000 7f4ea08c1700 1 -- 172.21.15.31:0/2739677948 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e98011a20 con 0x7f4e9c109040 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.330+0000 7f4ea3b27700 1 -- 172.21.15.31:0/2739677948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 msgr2=0x7f4e9c10b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.330+0000 7f4ea3b27700 1 --2- 172.21.15.31:0/2739677948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c10b430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4e980098a0 tx=0x7f4e98009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.330+0000 7f4ea3b27700 1 -- 172.21.15.31:0/2739677948 shutdown_connections 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.330+0000 7f4ea3b27700 1 --2- 172.21.15.31:0/2739677948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c10b430 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.330+0000 7f4ea3b27700 1 -- 172.21.15.31:0/2739677948 >> 172.21.15.31:0/2739677948 conn(0x7f4e9c102ad0 msgr2=0x7f4e9c104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.330+0000 7f4ea3b27700 1 -- 172.21.15.31:0/2739677948 shutdown_connections 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.330+0000 7f4ea3b27700 1 -- 172.21.15.31:0/2739677948 wait complete. 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.330+0000 7f4ea3b27700 1 Processor -- start 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4ea3b27700 1 -- start start 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4ea3b27700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c1088c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:37.394 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4ea3b27700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c10c980 con 0x7f4e9c109040 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4ea18c3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c1088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4ea18c3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c1088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59114/0 (socket says 172.21.15.31:59114) 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4ea18c3700 1 -- 172.21.15.31:0/1339883022 learned_addr learned my addr 172.21.15.31:0/1339883022 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4ea18c3700 1 -- 172.21.15.31:0/1339883022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c108e00 con 0x7f4e9c109040 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4ea18c3700 1 --2- 172.21.15.31:0/1339883022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c1088c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e98000c00 tx=0x7f4e98011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4e92ffd700 1 -- 172.21.15.31:0/1339883022 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e98003aa0 con 0x7f4e9c109040 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4e92ffd700 1 -- 172.21.15.31:0/1339883022 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e9801b8f0 con 0x7f4e9c109040 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4ea3b27700 1 -- 172.21.15.31:0/1339883022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9c106ff0 con 0x7f4e9c109040 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.331+0000 7f4e92ffd700 1 -- 172.21.15.31:0/1339883022 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e98023870 con 0x7f4e9c109040 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.332+0000 7f4ea3b27700 1 -- 172.21.15.31:0/1339883022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9c107530 con 0x7f4e9c109040 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.333+0000 7f4e92ffd700 1 -- 172.21.15.31:0/1339883022 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f4e98023b70 con 0x7f4e9c109040 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.333+0000 7f4ea3b27700 1 -- 172.21.15.31:0/1339883022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9c0640a0 con 0x7f4e9c109040 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.333+0000 7f4e92ffd700 1 --2- 172.21.15.31:0/1339883022 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f4e88039600 0x7f4e8803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:37.395 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.333+0000 7f4e92ffd700 1 -- 172.21.15.31:0/1339883022 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4e9804e150 con 0x7f4e9c109040 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.333+0000 7f4ea10c2700 1 --2- 172.21.15.31:0/1339883022 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f4e88039600 0x7f4e8803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.334+0000 7f4ea10c2700 1 --2- 172.21.15.31:0/1339883022 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f4e88039600 0x7f4e8803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c006fd0 tx=0x7f4e8c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.338+0000 7f4e92ffd700 1 -- 172.21.15.31:0/1339883022 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e9801f030 con 0x7f4e9c109040 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.568+0000 7f4ea3b27700 1 -- 172.21.15.31:0/1339883022 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f4e9c107cd0 con 0x7f4e88039600 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.816+0000 7f4e92ffd700 1 -- 172.21.15.31:0/1339883022 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f4e9c107cd0 con 0x7f4e88039600 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.818+0000 7f4ea3b27700 1 -- 172.21.15.31:0/1339883022 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f4e88039600 msgr2=0x7f4e8803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.818+0000 7f4ea3b27700 1 --2- 172.21.15.31:0/1339883022 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f4e88039600 0x7f4e8803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c006fd0 tx=0x7f4e8c006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.818+0000 7f4ea3b27700 1 -- 172.21.15.31:0/1339883022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 msgr2=0x7f4e9c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.818+0000 7f4ea3b27700 1 --2- 172.21.15.31:0/1339883022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c1088c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e98000c00 tx=0x7f4e98011aa0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.818+0000 7f4ea3b27700 1 -- 172.21.15.31:0/1339883022 shutdown_connections 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.818+0000 7f4ea3b27700 1 --2- 172.21.15.31:0/1339883022 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f4e88039600 0x7f4e8803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.818+0000 7f4ea3b27700 1 --2- 172.21.15.31:0/1339883022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c109040 0x7f4e9c1088c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:37.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.818+0000 7f4ea3b27700 1 -- 172.21.15.31:0/1339883022 >> 172.21.15.31:0/1339883022 conn(0x7f4e9c102ad0 msgr2=0x7f4e9c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:37.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.818+0000 7f4ea3b27700 1 -- 172.21.15.31:0/1339883022 shutdown_connections 2024-04-25T13:10:37.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:35.819+0000 7f4ea3b27700 1 -- 172.21.15.31:0/1339883022 wait complete. 2024-04-25T13:10:37.397 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2024-04-25T13:10:37.638 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:37 smithi031 ceph-mon[29457]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:38.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T13:10:38.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.818+0000 7f69267a4700 1 Processor -- start 2024-04-25T13:10:38.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.818+0000 7f69267a4700 1 -- start start 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.818+0000 7f69267a4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f6920107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.818+0000 7f69267a4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6920107ee0 con 0x7f69201074f0 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.818+0000 7f691ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f6920107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.818+0000 7f691ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f6920107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59130/0 (socket says 172.21.15.31:59130) 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.818+0000 7f691ffff700 1 -- 172.21.15.31:0/3741966207 learned_addr learned my addr 172.21.15.31:0/3741966207 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.818+0000 7f691ffff700 1 -- 172.21.15.31:0/3741966207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69201086f0 con 0x7f69201074f0 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.818+0000 7f691ffff700 1 --2- 172.21.15.31:0/3741966207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f6920107910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f69100098a0 tx=0x7f6910009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677ad1cc42ddbb5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.819+0000 7f691effd700 1 -- 172.21.15.31:0/3741966207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6910004160 con 0x7f69201074f0 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.819+0000 7f691effd700 1 -- 172.21.15.31:0/3741966207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6910004300 con 0x7f69201074f0 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.819+0000 7f691effd700 1 -- 172.21.15.31:0/3741966207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6910011a20 con 0x7f69201074f0 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.819+0000 7f69267a4700 1 -- 172.21.15.31:0/3741966207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 msgr2=0x7f6920107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.819+0000 7f69267a4700 1 --2- 172.21.15.31:0/3741966207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f6920107910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f69100098a0 tx=0x7f6910009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.819+0000 7f69267a4700 1 -- 172.21.15.31:0/3741966207 shutdown_connections 2024-04-25T13:10:38.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.819+0000 7f69267a4700 1 --2- 172.21.15.31:0/3741966207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f6920107910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.819+0000 7f69267a4700 1 -- 172.21.15.31:0/3741966207 >> 172.21.15.31:0/3741966207 conn(0x7f6920102a90 msgr2=0x7f6920104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.819+0000 7f69267a4700 1 -- 172.21.15.31:0/3741966207 shutdown_connections 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.819+0000 7f69267a4700 1 -- 172.21.15.31:0/3741966207 wait complete. 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.820+0000 7f69267a4700 1 Processor -- start 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.820+0000 7f69267a4700 1 -- start start 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.820+0000 7f69267a4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f692019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.820+0000 7f69267a4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f692010baa0 con 0x7f69201074f0 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.820+0000 7f691ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f692019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.820+0000 7f691ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f692019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59142/0 (socket says 172.21.15.31:59142) 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.820+0000 7f691ffff700 1 -- 172.21.15.31:0/3136915504 learned_addr learned my addr 172.21.15.31:0/3136915504 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.821+0000 7f691ffff700 1 -- 172.21.15.31:0/3136915504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692019c9f0 con 0x7f69201074f0 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.821+0000 7f691ffff700 1 --2- 172.21.15.31:0/3136915504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f692019c4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6910009520 tx=0x7f6910011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.821+0000 7f691d7fa700 1 -- 172.21.15.31:0/3136915504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69100039e0 con 0x7f69201074f0 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.821+0000 7f691d7fa700 1 -- 172.21.15.31:0/3136915504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691001b8f0 con 0x7f69201074f0 2024-04-25T13:10:38.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.821+0000 7f69267a4700 1 -- 172.21.15.31:0/3136915504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692019cc80 con 0x7f69201074f0 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.821+0000 7f691d7fa700 1 -- 172.21.15.31:0/3136915504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6910019770 con 0x7f69201074f0 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.821+0000 7f69267a4700 1 -- 172.21.15.31:0/3136915504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692019f920 con 0x7f69201074f0 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.822+0000 7f691d7fa700 1 -- 172.21.15.31:0/3136915504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f6910019a20 con 0x7f69201074f0 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.822+0000 7f69267a4700 1 -- 172.21.15.31:0/3136915504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692010bc20 con 0x7f69201074f0 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.822+0000 7f691d7fa700 1 --2- 172.21.15.31:0/3136915504 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f690c039650 0x7f690c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.822+0000 7f691d7fa700 1 -- 172.21.15.31:0/3136915504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f691004dfe0 con 0x7f69201074f0 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.822+0000 7f691f7fe700 1 --2- 172.21.15.31:0/3136915504 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f690c039650 0x7f690c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.823+0000 7f691f7fe700 1 --2- 172.21.15.31:0/3136915504 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f690c039650 0x7f690c03bb10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6908006fd0 tx=0x7f6908006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:37.827+0000 7f691d7fa700 1 -- 172.21.15.31:0/3136915504 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f691001f030 con 0x7f69201074f0 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.033+0000 7f69267a4700 1 -- 172.21.15.31:0/3136915504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f692002e140 con 0x7f69201074f0 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.033+0000 7f691d7fa700 1 -- 172.21.15.31:0/3136915504 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f69100172b0 con 0x7f69201074f0 2024-04-25T13:10:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.035+0000 7f69267a4700 1 -- 172.21.15.31:0/3136915504 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f690c039650 msgr2=0x7f690c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:38.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.035+0000 7f69267a4700 1 --2- 172.21.15.31:0/3136915504 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f690c039650 0x7f690c03bb10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6908006fd0 tx=0x7f6908006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:38.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.035+0000 7f69267a4700 1 -- 172.21.15.31:0/3136915504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 msgr2=0x7f692019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:38.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.035+0000 7f69267a4700 1 --2- 172.21.15.31:0/3136915504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f692019c4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6910009520 tx=0x7f6910011aa0 comp rx=0 tx=0).stop 2024-04-25T13:10:38.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.035+0000 7f69267a4700 1 -- 172.21.15.31:0/3136915504 shutdown_connections 2024-04-25T13:10:38.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.035+0000 7f69267a4700 1 --2- 172.21.15.31:0/3136915504 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f690c039650 0x7f690c03bb10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:38.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.035+0000 7f69267a4700 1 --2- 172.21.15.31:0/3136915504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69201074f0 0x7f692019c4b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:38.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.035+0000 7f69267a4700 1 -- 172.21.15.31:0/3136915504 >> 172.21.15.31:0/3136915504 conn(0x7f6920102a90 msgr2=0x7f6920103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:38.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.036+0000 7f69267a4700 1 -- 172.21.15.31:0/3136915504 shutdown_connections 2024-04-25T13:10:38.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.036+0000 7f69267a4700 1 -- 172.21.15.31:0/3136915504 wait complete. 2024-04-25T13:10:38.516 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:10:38.516 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-25T13:10:38.524 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:10:38.524 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-25T13:10:38.524 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-25T13:10:38.524 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T13:10:38.526 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2024-04-25T13:10:38.526 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:10:38.526 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031:8443/ 2024-04-25T13:10:38.526 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2024-04-25T13:10:38.526 INFO:teuthology.orchestra.run.smithi031.stdout: Password: zl43vd1cb1 2024-04-25T13:10:38.526 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:10:38.527 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config directory 2024-04-25T13:10:38.527 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling autotune for osd_memory_target 2024-04-25T13:10:38.650 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:38 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3136915504' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T13:10:39.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:39 smithi031 ceph-mon[29457]: mgrmap e13: x(active, since 6s) 2024-04-25T13:10:39.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.985+0000 7f0d48d40700 1 Processor -- start 2024-04-25T13:10:39.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.985+0000 7f0d48d40700 1 -- start start 2024-04-25T13:10:39.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.986+0000 7f0d48d40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d441078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:39.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.986+0000 7f0d48d40700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44107ea0 con 0x7f0d441074b0 2024-04-25T13:10:39.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.986+0000 7f0d4259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d441078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:39.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.986+0000 7f0d4259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d441078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59144/0 (socket says 172.21.15.31:59144) 2024-04-25T13:10:39.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.986+0000 7f0d4259c700 1 -- 172.21.15.31:0/894402416 learned_addr learned my addr 172.21.15.31:0/894402416 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:39.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.986+0000 7f0d4259c700 1 -- 172.21.15.31:0/894402416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d441086b0 con 0x7f0d441074b0 2024-04-25T13:10:39.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.986+0000 7f0d4259c700 1 --2- 172.21.15.31:0/894402416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d441078d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400bb40 tx=0x7f0d3400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f9eefcfd8bdd96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:39.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.987+0000 7f0d4159a700 1 -- 172.21.15.31:0/894402416 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3400dbe0 con 0x7f0d441074b0 2024-04-25T13:10:39.765 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.987+0000 7f0d4159a700 1 -- 172.21.15.31:0/894402416 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d340044e0 con 0x7f0d441074b0 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.987+0000 7f0d4159a700 1 -- 172.21.15.31:0/894402416 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d34010470 con 0x7f0d441074b0 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.987+0000 7f0d48d40700 1 -- 172.21.15.31:0/894402416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 msgr2=0x7f0d441078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.987+0000 7f0d48d40700 1 --2- 172.21.15.31:0/894402416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d441078d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400bb40 tx=0x7f0d3400be50 comp rx=0 tx=0).stop 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.987+0000 7f0d48d40700 1 -- 172.21.15.31:0/894402416 shutdown_connections 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.987+0000 7f0d48d40700 1 --2- 172.21.15.31:0/894402416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d441078d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.987+0000 7f0d48d40700 1 -- 172.21.15.31:0/894402416 >> 172.21.15.31:0/894402416 conn(0x7f0d44102ab0 msgr2=0x7f0d44104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.987+0000 7f0d48d40700 1 -- 172.21.15.31:0/894402416 shutdown_connections 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.987+0000 7f0d48d40700 1 -- 172.21.15.31:0/894402416 wait complete. 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.988+0000 7f0d48d40700 1 Processor -- start 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.988+0000 7f0d48d40700 1 -- start start 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.988+0000 7f0d48d40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d4419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.988+0000 7f0d48d40700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4407afb0 con 0x7f0d441074b0 2024-04-25T13:10:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.988+0000 7f0d4259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d4419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:39.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.988+0000 7f0d4259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d4419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59156/0 (socket says 172.21.15.31:59156) 2024-04-25T13:10:39.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.988+0000 7f0d4259c700 1 -- 172.21.15.31:0/2582317951 learned_addr learned my addr 172.21.15.31:0/2582317951 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:39.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.989+0000 7f0d4259c700 1 -- 172.21.15.31:0/2582317951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4419a7f0 con 0x7f0d441074b0 2024-04-25T13:10:39.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.989+0000 7f0d4259c700 1 --2- 172.21.15.31:0/2582317951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d4419a2b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d34003c30 tx=0x7f0d34003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:39.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.989+0000 7f0d337fe700 1 -- 172.21.15.31:0/2582317951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d34027070 con 0x7f0d441074b0 2024-04-25T13:10:39.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.989+0000 7f0d337fe700 1 -- 172.21.15.31:0/2582317951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d34010bd0 con 0x7f0d441074b0 2024-04-25T13:10:39.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.989+0000 7f0d48d40700 1 -- 172.21.15.31:0/2582317951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4419aa80 con 0x7f0d441074b0 2024-04-25T13:10:39.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.989+0000 7f0d337fe700 1 -- 172.21.15.31:0/2582317951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d34022a20 con 0x7f0d441074b0 2024-04-25T13:10:39.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.989+0000 7f0d48d40700 1 -- 172.21.15.31:0/2582317951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4419d720 con 0x7f0d441074b0 2024-04-25T13:10:39.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.990+0000 7f0d337fe700 1 -- 172.21.15.31:0/2582317951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0d3402a070 con 0x7f0d441074b0 2024-04-25T13:10:39.767 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.990+0000 7f0d48d40700 1 -- 172.21.15.31:0/2582317951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d440640a0 con 0x7f0d441074b0 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.990+0000 7f0d337fe700 1 --2- 172.21.15.31:0/2582317951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0d2c0396a0 0x7f0d2c03bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.990+0000 7f0d337fe700 1 -- 172.21.15.31:0/2582317951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0d3401f070 con 0x7f0d441074b0 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.990+0000 7f0d41d9b700 1 --2- 172.21.15.31:0/2582317951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0d2c0396a0 0x7f0d2c03bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.991+0000 7f0d41d9b700 1 --2- 172.21.15.31:0/2582317951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0d2c0396a0 0x7f0d2c03bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0d38006fd0 tx=0x7f0d38006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:38.995+0000 7f0d337fe700 1 -- 172.21.15.31:0/2582317951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d34015070 con 0x7f0d441074b0 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.202+0000 7f0d48d40700 1 -- 172.21.15.31:0/2582317951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f0d44193810 con 0x7f0d441074b0 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.202+0000 7f0d337fe700 1 -- 172.21.15.31:0/2582317951 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f0d44193810 con 0x7f0d441074b0 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.204+0000 7f0d48d40700 1 -- 172.21.15.31:0/2582317951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0d2c0396a0 msgr2=0x7f0d2c03bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.204+0000 7f0d48d40700 1 --2- 172.21.15.31:0/2582317951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0d2c0396a0 0x7f0d2c03bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0d38006fd0 tx=0x7f0d38006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.204+0000 7f0d48d40700 1 -- 172.21.15.31:0/2582317951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 msgr2=0x7f0d4419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.204+0000 7f0d48d40700 1 --2- 172.21.15.31:0/2582317951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d4419a2b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d34003c30 tx=0x7f0d34003d10 comp rx=0 tx=0).stop 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.204+0000 7f0d48d40700 1 -- 172.21.15.31:0/2582317951 shutdown_connections 2024-04-25T13:10:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.204+0000 7f0d48d40700 1 --2- 172.21.15.31:0/2582317951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0d2c0396a0 0x7f0d2c03bb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.204+0000 7f0d48d40700 1 --2- 172.21.15.31:0/2582317951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d441074b0 0x7f0d4419a2b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:39.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.204+0000 7f0d48d40700 1 -- 172.21.15.31:0/2582317951 >> 172.21.15.31:0/2582317951 conn(0x7f0d44102ab0 msgr2=0x7f0d441908d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:39.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.204+0000 7f0d48d40700 1 -- 172.21.15.31:0/2582317951 shutdown_connections 2024-04-25T13:10:39.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:39.204+0000 7f0d48d40700 1 -- 172.21.15.31:0/2582317951 wait complete. 2024-04-25T13:10:41.083 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.165+0000 7f7fd4fde700 1 Processor -- start 2024-04-25T13:10:41.083 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.165+0000 7f7fd4fde700 1 -- start start 2024-04-25T13:10:41.083 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.166+0000 7f7fd4fde700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd01089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:41.084 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.166+0000 7f7fd4fde700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd0108fc0 con 0x7f7fd01085d0 2024-04-25T13:10:41.084 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.166+0000 7f7fce59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd01089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:41.084 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.166+0000 7f7fce59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd01089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59160/0 (socket says 172.21.15.31:59160) 2024-04-25T13:10:41.084 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.166+0000 7f7fce59c700 1 -- 172.21.15.31:0/1233359128 learned_addr learned my addr 172.21.15.31:0/1233359128 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:41.084 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.166+0000 7f7fce59c700 1 -- 172.21.15.31:0/1233359128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd01097d0 con 0x7f7fd01085d0 2024-04-25T13:10:41.085 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.166+0000 7f7fce59c700 1 --2- 172.21.15.31:0/1233359128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd01089f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00098a0 tx=0x7f7fc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d36cd169ccb8446 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:41.085 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.166+0000 7f7fcd59a700 1 -- 172.21.15.31:0/1233359128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fc0004160 con 0x7f7fd01085d0 2024-04-25T13:10:41.085 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.167+0000 7f7fcd59a700 1 -- 172.21.15.31:0/1233359128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc0004300 con 0x7f7fd01085d0 2024-04-25T13:10:41.085 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.167+0000 7f7fcd59a700 1 -- 172.21.15.31:0/1233359128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fc0011a20 con 0x7f7fd01085d0 2024-04-25T13:10:41.085 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.167+0000 7f7fd4fde700 1 -- 172.21.15.31:0/1233359128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 msgr2=0x7f7fd01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:41.085 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.167+0000 7f7fd4fde700 1 --2- 172.21.15.31:0/1233359128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd01089f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00098a0 tx=0x7f7fc0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:10:41.085 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.167+0000 7f7fd4fde700 1 -- 172.21.15.31:0/1233359128 shutdown_connections 2024-04-25T13:10:41.085 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.167+0000 7f7fd4fde700 1 --2- 172.21.15.31:0/1233359128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd01089f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:41.085 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.167+0000 7f7fd4fde700 1 -- 172.21.15.31:0/1233359128 >> 172.21.15.31:0/1233359128 conn(0x7f7fd0103bf0 msgr2=0x7f7fd0106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:41.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.167+0000 7f7fd4fde700 1 -- 172.21.15.31:0/1233359128 shutdown_connections 2024-04-25T13:10:41.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.167+0000 7f7fd4fde700 1 -- 172.21.15.31:0/1233359128 wait complete. 2024-04-25T13:10:41.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.168+0000 7f7fd4fde700 1 Processor -- start 2024-04-25T13:10:41.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.168+0000 7f7fd4fde700 1 -- start start 2024-04-25T13:10:41.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.168+0000 7f7fd4fde700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:41.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.168+0000 7f7fd4fde700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd010cb80 con 0x7f7fd01085d0 2024-04-25T13:10:41.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.168+0000 7f7fce59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:41.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.168+0000 7f7fce59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59162/0 (socket says 172.21.15.31:59162) 2024-04-25T13:10:41.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.168+0000 7f7fce59c700 1 -- 172.21.15.31:0/269990300 learned_addr learned my addr 172.21.15.31:0/269990300 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:41.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.169+0000 7f7fce59c700 1 -- 172.21.15.31:0/269990300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd019eb80 con 0x7f7fd01085d0 2024-04-25T13:10:41.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.169+0000 7f7fce59c700 1 --2- 172.21.15.31:0/269990300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd019e640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00116e0 tx=0x7f7fc0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:41.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.169+0000 7f7fbf7fe700 1 -- 172.21.15.31:0/269990300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fc00039e0 con 0x7f7fd01085d0 2024-04-25T13:10:41.088 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.169+0000 7f7fd4fde700 1 -- 172.21.15.31:0/269990300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd019ee10 con 0x7f7fd01085d0 2024-04-25T13:10:41.088 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.169+0000 7f7fbf7fe700 1 -- 172.21.15.31:0/269990300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc001b8f0 con 0x7f7fd01085d0 2024-04-25T13:10:41.088 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.169+0000 7f7fbf7fe700 1 -- 172.21.15.31:0/269990300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fc0023760 con 0x7f7fd01085d0 2024-04-25T13:10:41.088 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.169+0000 7f7fd4fde700 1 -- 172.21.15.31:0/269990300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd01a1b00 con 0x7f7fd01085d0 2024-04-25T13:10:41.088 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.170+0000 7f7fbf7fe700 1 -- 172.21.15.31:0/269990300 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7fc00239f0 con 0x7f7fd01085d0 2024-04-25T13:10:41.088 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.170+0000 7f7fbf7fe700 1 --2- 172.21.15.31:0/269990300 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7fb8039650 0x7f7fb803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:41.089 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.170+0000 7f7fbf7fe700 1 -- 172.21.15.31:0/269990300 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7fc004e130 con 0x7f7fd01085d0 2024-04-25T13:10:41.089 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.170+0000 7f7fd4fde700 1 -- 172.21.15.31:0/269990300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd010cd00 con 0x7f7fd01085d0 2024-04-25T13:10:41.089 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.170+0000 7f7fcdd9b700 1 --2- 172.21.15.31:0/269990300 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7fb8039650 0x7f7fb803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:41.089 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.171+0000 7f7fcdd9b700 1 --2- 172.21.15.31:0/269990300 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7fb8039650 0x7f7fb803bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4006fd0 tx=0x7f7fc4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:41.089 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.175+0000 7f7fbf7fe700 1 -- 172.21.15.31:0/269990300 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fc001f030 con 0x7f7fd01085d0 2024-04-25T13:10:41.090 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.497+0000 7f7fd4fde700 1 -- 172.21.15.31:0/269990300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f7fd00640a0 con 0x7f7fd01085d0 2024-04-25T13:10:41.090 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.499+0000 7f7fbf7fe700 1 -- 172.21.15.31:0/269990300 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f7fc0018c10 con 0x7f7fd01085d0 2024-04-25T13:10:41.090 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.502+0000 7f7fd4fde700 1 -- 172.21.15.31:0/269990300 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7fb8039650 msgr2=0x7f7fb803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:41.090 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.502+0000 7f7fd4fde700 1 --2- 172.21.15.31:0/269990300 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7fb8039650 0x7f7fb803bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4006fd0 tx=0x7f7fc4006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:41.090 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.502+0000 7f7fd4fde700 1 -- 172.21.15.31:0/269990300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 msgr2=0x7f7fd019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:41.090 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.502+0000 7f7fd4fde700 1 --2- 172.21.15.31:0/269990300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd019e640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00116e0 tx=0x7f7fc0011aa0 comp rx=0 tx=0).stop 2024-04-25T13:10:41.091 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.502+0000 7f7fd4fde700 1 -- 172.21.15.31:0/269990300 shutdown_connections 2024-04-25T13:10:41.091 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.502+0000 7f7fd4fde700 1 --2- 172.21.15.31:0/269990300 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7fb8039650 0x7f7fb803bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:41.091 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.502+0000 7f7fd4fde700 1 --2- 172.21.15.31:0/269990300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01085d0 0x7f7fd019e640 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:41.091 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.502+0000 7f7fd4fde700 1 -- 172.21.15.31:0/269990300 >> 172.21.15.31:0/269990300 conn(0x7f7fd0103bf0 msgr2=0x7f7fd0104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:41.091 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.502+0000 7f7fd4fde700 1 -- 172.21.15.31:0/269990300 shutdown_connections 2024-04-25T13:10:41.091 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-25T13:10:40.502+0000 7f7fd4fde700 1 -- 172.21.15.31:0/269990300 wait complete. 2024-04-25T13:10:41.092 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T13:10:41.092 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T13:10:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:10:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:10:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:10:41.092 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T13:10:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:10:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T13:10:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:10:41.093 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T13:10:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:10:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2024-04-25T13:10:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:10:41.094 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2024-04-25T13:10:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:10:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T13:10:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:10:41.094 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2024-04-25T13:10:41.121 INFO:tasks.cephadm:Fetching config... 2024-04-25T13:10:41.121 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:10:41.121 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T13:10:41.138 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T13:10:41.138 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:10:41.138 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T13:10:41.194 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T13:10:41.194 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:10:41.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T13:10:41.259 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T13:10:41.259 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:10:41.259 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T13:10:41.318 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T13:10:41.318 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd21Kz63tEiVZqeHg2TtSIOO6/5WQ4oj7DOv9bHDL/XMY6D16abVmSEXoymVZqjP1yBG/NaJX0Brcfo+PCwIzHR4j92uHXqJoNDqaSGtFK2FevzzfdZwzUUo6VKuO1036ak9Sy5ly9Hg7fOUtGiGHdXdr/XfVXyrNh+Rc1WiZqItdmtL/eeR1b71UHHSXJLGAP2b1m6Zmbw/M/Gec0oCxKwX3TOq/dmrKpfJaVDWE4qpGRa4E2MRpTb+0tf0c1UkKChlOYWgfwMZe/NQsHoznSbNxaS1EZVjuArk/279ggX23RcjtMDpS8QgCV5/zDAU122VJHQR0Y0pw9DcWgy20ABD3DGA3TgAVsUG4Q4RjMrY3rJd4lfp/fMFhfkVOFimCyZHq/m7jV03lWmn0aXQxl9kGMnQzNUpRp6mZK1UObkyx9niHY231WN3poji7BFeJgqBR1rdXtc33GI5NnRcOAPOi6yMTlCKp1yWj1w3aw3qcE4QRCCWvjCkqnUHVOB80= ceph-e4b023f4-0304-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:10:41.424 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd21Kz63tEiVZqeHg2TtSIOO6/5WQ4oj7DOv9bHDL/XMY6D16abVmSEXoymVZqjP1yBG/NaJX0Brcfo+PCwIzHR4j92uHXqJoNDqaSGtFK2FevzzfdZwzUUo6VKuO1036ak9Sy5ly9Hg7fOUtGiGHdXdr/XfVXyrNh+Rc1WiZqItdmtL/eeR1b71UHHSXJLGAP2b1m6Zmbw/M/Gec0oCxKwX3TOq/dmrKpfJaVDWE4qpGRa4E2MRpTb+0tf0c1UkKChlOYWgfwMZe/NQsHoznSbNxaS1EZVjuArk/279ggX23RcjtMDpS8QgCV5/zDAU122VJHQR0Y0pw9DcWgy20ABD3DGA3TgAVsUG4Q4RjMrY3rJd4lfp/fMFhfkVOFimCyZHq/m7jV03lWmn0aXQxl9kGMnQzNUpRp6mZK1UObkyx9niHY231WN3poji7BFeJgqBR1rdXtc33GI5NnRcOAPOi6yMTlCKp1yWj1w3aw3qcE4QRCCWvjCkqnUHVOB80= ceph-e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:10:41.436 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd21Kz63tEiVZqeHg2TtSIOO6/5WQ4oj7DOv9bHDL/XMY6D16abVmSEXoymVZqjP1yBG/NaJX0Brcfo+PCwIzHR4j92uHXqJoNDqaSGtFK2FevzzfdZwzUUo6VKuO1036ak9Sy5ly9Hg7fOUtGiGHdXdr/XfVXyrNh+Rc1WiZqItdmtL/eeR1b71UHHSXJLGAP2b1m6Zmbw/M/Gec0oCxKwX3TOq/dmrKpfJaVDWE4qpGRa4E2MRpTb+0tf0c1UkKChlOYWgfwMZe/NQsHoznSbNxaS1EZVjuArk/279ggX23RcjtMDpS8QgCV5/zDAU122VJHQR0Y0pw9DcWgy20ABD3DGA3TgAVsUG4Q4RjMrY3rJd4lfp/fMFhfkVOFimCyZHq/m7jV03lWmn0aXQxl9kGMnQzNUpRp6mZK1UObkyx9niHY231WN3poji7BFeJgqBR1rdXtc33GI5NnRcOAPOi6yMTlCKp1yWj1w3aw3qcE4QRCCWvjCkqnUHVOB80= ceph-e4b023f4-0304-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:10:41.478 INFO:teuthology.orchestra.run.smithi066.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd21Kz63tEiVZqeHg2TtSIOO6/5WQ4oj7DOv9bHDL/XMY6D16abVmSEXoymVZqjP1yBG/NaJX0Brcfo+PCwIzHR4j92uHXqJoNDqaSGtFK2FevzzfdZwzUUo6VKuO1036ak9Sy5ly9Hg7fOUtGiGHdXdr/XfVXyrNh+Rc1WiZqItdmtL/eeR1b71UHHSXJLGAP2b1m6Zmbw/M/Gec0oCxKwX3TOq/dmrKpfJaVDWE4qpGRa4E2MRpTb+0tf0c1UkKChlOYWgfwMZe/NQsHoznSbNxaS1EZVjuArk/279ggX23RcjtMDpS8QgCV5/zDAU122VJHQR0Y0pw9DcWgy20ABD3DGA3TgAVsUG4Q4RjMrY3rJd4lfp/fMFhfkVOFimCyZHq/m7jV03lWmn0aXQxl9kGMnQzNUpRp6mZK1UObkyx9niHY231WN3poji7BFeJgqBR1rdXtc33GI5NnRcOAPOi6yMTlCKp1yWj1w3aw3qcE4QRCCWvjCkqnUHVOB80= ceph-e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:10:41.491 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd21Kz63tEiVZqeHg2TtSIOO6/5WQ4oj7DOv9bHDL/XMY6D16abVmSEXoymVZqjP1yBG/NaJX0Brcfo+PCwIzHR4j92uHXqJoNDqaSGtFK2FevzzfdZwzUUo6VKuO1036ak9Sy5ly9Hg7fOUtGiGHdXdr/XfVXyrNh+Rc1WiZqItdmtL/eeR1b71UHHSXJLGAP2b1m6Zmbw/M/Gec0oCxKwX3TOq/dmrKpfJaVDWE4qpGRa4E2MRpTb+0tf0c1UkKChlOYWgfwMZe/NQsHoznSbNxaS1EZVjuArk/279ggX23RcjtMDpS8QgCV5/zDAU122VJHQR0Y0pw9DcWgy20ABD3DGA3TgAVsUG4Q4RjMrY3rJd4lfp/fMFhfkVOFimCyZHq/m7jV03lWmn0aXQxl9kGMnQzNUpRp6mZK1UObkyx9niHY231WN3poji7BFeJgqBR1rdXtc33GI5NnRcOAPOi6yMTlCKp1yWj1w3aw3qcE4QRCCWvjCkqnUHVOB80= ceph-e4b023f4-0304-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:10:41.530 INFO:teuthology.orchestra.run.smithi136.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDd21Kz63tEiVZqeHg2TtSIOO6/5WQ4oj7DOv9bHDL/XMY6D16abVmSEXoymVZqjP1yBG/NaJX0Brcfo+PCwIzHR4j92uHXqJoNDqaSGtFK2FevzzfdZwzUUo6VKuO1036ak9Sy5ly9Hg7fOUtGiGHdXdr/XfVXyrNh+Rc1WiZqItdmtL/eeR1b71UHHSXJLGAP2b1m6Zmbw/M/Gec0oCxKwX3TOq/dmrKpfJaVDWE4qpGRa4E2MRpTb+0tf0c1UkKChlOYWgfwMZe/NQsHoznSbNxaS1EZVjuArk/279ggX23RcjtMDpS8QgCV5/zDAU122VJHQR0Y0pw9DcWgy20ABD3DGA3TgAVsUG4Q4RjMrY3rJd4lfp/fMFhfkVOFimCyZHq/m7jV03lWmn0aXQxl9kGMnQzNUpRp6mZK1UObkyx9niHY231WN3poji7BFeJgqBR1rdXtc33GI5NnRcOAPOi6yMTlCKp1yWj1w3aw3qcE4QRCCWvjCkqnUHVOB80= ceph-e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:10:41.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T13:10:41.657 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:41 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/269990300' entity='client.admin' 2024-04-25T13:10:41.797 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:10:44.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.258+0000 7fd64155c700 1 -- 172.21.15.31:0/2322720379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c1013b0 msgr2=0x7fd63c1017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:44.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.258+0000 7fd64155c700 1 --2- 172.21.15.31:0/2322720379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c1013b0 0x7fd63c1017d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009910 tx=0x7fd62c009c20 comp rx=0 tx=0).stop 2024-04-25T13:10:44.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.258+0000 7fd64155c700 1 -- 172.21.15.31:0/2322720379 shutdown_connections 2024-04-25T13:10:44.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.258+0000 7fd64155c700 1 --2- 172.21.15.31:0/2322720379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c1013b0 0x7fd63c1017d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.258+0000 7fd64155c700 1 -- 172.21.15.31:0/2322720379 >> 172.21.15.31:0/2322720379 conn(0x7fd63c0fc990 msgr2=0x7fd63c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:44.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.258+0000 7fd64155c700 1 -- 172.21.15.31:0/2322720379 shutdown_connections 2024-04-25T13:10:44.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.258+0000 7fd64155c700 1 -- 172.21.15.31:0/2322720379 wait complete. 2024-04-25T13:10:44.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.259+0000 7fd64155c700 1 Processor -- start 2024-04-25T13:10:44.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.259+0000 7fd64155c700 1 -- start start 2024-04-25T13:10:44.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.259+0000 7fd64155c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c1013b0 0x7fd63c0fe800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:44.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.259+0000 7fd64155c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1072f0 con 0x7fd63c1013b0 2024-04-25T13:10:44.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.259+0000 7fd63affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c1013b0 0x7fd63c0fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:44.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.259+0000 7fd63affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c1013b0 0x7fd63c0fe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36420/0 (socket says 172.21.15.31:36420) 2024-04-25T13:10:44.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.259+0000 7fd63affd700 1 -- 172.21.15.31:0/715487006 learned_addr learned my addr 172.21.15.31:0/715487006 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:44.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.260+0000 7fd63affd700 1 -- 172.21.15.31:0/715487006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c0fed40 con 0x7fd63c1013b0 2024-04-25T13:10:44.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.260+0000 7fd63affd700 1 --2- 172.21.15.31:0/715487006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c1013b0 0x7fd63c0fe800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005950 tx=0x7fd62c004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:44.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.260+0000 7fd623fff700 1 -- 172.21.15.31:0/715487006 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd62c026070 con 0x7fd63c1013b0 2024-04-25T13:10:44.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.260+0000 7fd623fff700 1 -- 172.21.15.31:0/715487006 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62c00bc00 con 0x7fd63c1013b0 2024-04-25T13:10:44.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.260+0000 7fd64155c700 1 -- 172.21.15.31:0/715487006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c0fcf30 con 0x7fd63c1013b0 2024-04-25T13:10:44.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.260+0000 7fd623fff700 1 -- 172.21.15.31:0/715487006 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd62c003a10 con 0x7fd63c1013b0 2024-04-25T13:10:44.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.260+0000 7fd64155c700 1 -- 172.21.15.31:0/715487006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c0fd470 con 0x7fd63c1013b0 2024-04-25T13:10:44.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.261+0000 7fd623fff700 1 -- 172.21.15.31:0/715487006 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd62c029070 con 0x7fd63c1013b0 2024-04-25T13:10:44.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.261+0000 7fd64155c700 1 -- 172.21.15.31:0/715487006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c107a70 con 0x7fd63c1013b0 2024-04-25T13:10:44.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.261+0000 7fd623fff700 1 --2- 172.21.15.31:0/715487006 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd6240396b0 0x7fd62403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:44.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.261+0000 7fd623fff700 1 -- 172.21.15.31:0/715487006 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd62c01e070 con 0x7fd63c1013b0 2024-04-25T13:10:44.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.261+0000 7fd63a7fc700 1 --2- 172.21.15.31:0/715487006 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd6240396b0 0x7fd62403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:44.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.262+0000 7fd63a7fc700 1 --2- 172.21.15.31:0/715487006 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd6240396b0 0x7fd62403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd630006fd0 tx=0x7fd630006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:44.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.267+0000 7fd623fff700 1 -- 172.21.15.31:0/715487006 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd62c013070 con 0x7fd63c1013b0 2024-04-25T13:10:44.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.476+0000 7fd64155c700 1 -- 172.21.15.31:0/715487006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fd63c191040 con 0x7fd63c1013b0 2024-04-25T13:10:44.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.479+0000 7fd623fff700 1 -- 172.21.15.31:0/715487006 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fd62c030020 con 0x7fd63c1013b0 2024-04-25T13:10:44.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.482+0000 7fd64155c700 1 -- 172.21.15.31:0/715487006 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd6240396b0 msgr2=0x7fd62403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:44.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.482+0000 7fd64155c700 1 --2- 172.21.15.31:0/715487006 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd6240396b0 0x7fd62403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd630006fd0 tx=0x7fd630006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:44.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.482+0000 7fd64155c700 1 -- 172.21.15.31:0/715487006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c1013b0 msgr2=0x7fd63c0fe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:44.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.482+0000 7fd64155c700 1 --2- 172.21.15.31:0/715487006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c1013b0 0x7fd63c0fe800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005950 tx=0x7fd62c004fa0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.482+0000 7fd64155c700 1 -- 172.21.15.31:0/715487006 shutdown_connections 2024-04-25T13:10:44.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.482+0000 7fd64155c700 1 --2- 172.21.15.31:0/715487006 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd6240396b0 0x7fd62403bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.482+0000 7fd64155c700 1 --2- 172.21.15.31:0/715487006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c1013b0 0x7fd63c0fe800 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:44.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.482+0000 7fd64155c700 1 -- 172.21.15.31:0/715487006 >> 172.21.15.31:0/715487006 conn(0x7fd63c0fc990 msgr2=0x7fd63c06ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:44.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.482+0000 7fd64155c700 1 -- 172.21.15.31:0/715487006 shutdown_connections 2024-04-25T13:10:44.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:44.482+0000 7fd64155c700 1 -- 172.21.15.31:0/715487006 wait complete. 2024-04-25T13:10:45.216 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T13:10:45.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T13:10:45.692 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:10:45.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:45 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/715487006' entity='client.admin' 2024-04-25T13:10:46.914 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:46.914 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:46.914 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:10:46.914 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:46.914 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:46.915 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:47.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.230+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2212605034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd4102f80 msgr2=0x7f0bd4105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:47.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.230+0000 7f0bdc3ea700 1 --2- 172.21.15.31:0/2212605034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd4102f80 0x7f0bd4105370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009910 tx=0x7f0bd0009c20 comp rx=0 tx=0).stop 2024-04-25T13:10:47.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.230+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2212605034 shutdown_connections 2024-04-25T13:10:47.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.230+0000 7f0bdc3ea700 1 --2- 172.21.15.31:0/2212605034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd4102f80 0x7f0bd4105370 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:47.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.230+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2212605034 >> 172.21.15.31:0/2212605034 conn(0x7f0bd40fc950 msgr2=0x7f0bd40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:47.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.230+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2212605034 shutdown_connections 2024-04-25T13:10:47.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.230+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2212605034 wait complete. 2024-04-25T13:10:47.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.231+0000 7f0bdc3ea700 1 Processor -- start 2024-04-25T13:10:47.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.231+0000 7f0bdc3ea700 1 -- start start 2024-04-25T13:10:47.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.231+0000 7f0bdc3ea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd4102f80 0x7f0bd4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:47.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.231+0000 7f0bdc3ea700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41083d0 con 0x7f0bd4102f80 2024-04-25T13:10:47.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.231+0000 7f0bda186700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd4102f80 0x7f0bd4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:47.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.231+0000 7f0bda186700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd4102f80 0x7f0bd4197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36436/0 (socket says 172.21.15.31:36436) 2024-04-25T13:10:47.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.231+0000 7f0bda186700 1 -- 172.21.15.31:0/2688295903 learned_addr learned my addr 172.21.15.31:0/2688295903 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:47.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.232+0000 7f0bda186700 1 -- 172.21.15.31:0/2688295903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd4197f80 con 0x7f0bd4102f80 2024-04-25T13:10:47.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.232+0000 7f0bda186700 1 --2- 172.21.15.31:0/2688295903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd4102f80 0x7f0bd4197a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005b40 tx=0x7f0bd0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:47.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.232+0000 7f0bcb7fe700 1 -- 172.21.15.31:0/2688295903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bd0027070 con 0x7f0bd4102f80 2024-04-25T13:10:47.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.232+0000 7f0bcb7fe700 1 -- 172.21.15.31:0/2688295903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd000be10 con 0x7f0bd4102f80 2024-04-25T13:10:47.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.232+0000 7f0bcb7fe700 1 -- 172.21.15.31:0/2688295903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bd0022de0 con 0x7f0bd4102f80 2024-04-25T13:10:47.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.232+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2688295903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd4198270 con 0x7f0bd4102f80 2024-04-25T13:10:47.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.232+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2688295903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd419af00 con 0x7f0bd4102f80 2024-04-25T13:10:47.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.233+0000 7f0bcb7fe700 1 -- 172.21.15.31:0/2688295903 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0bd002a070 con 0x7f0bd4102f80 2024-04-25T13:10:47.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.233+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2688295903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd4108b50 con 0x7f0bd4102f80 2024-04-25T13:10:47.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.233+0000 7f0bcb7fe700 1 --2- 172.21.15.31:0/2688295903 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0bc00396b0 0x7f0bc003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:47.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.233+0000 7f0bcb7fe700 1 -- 172.21.15.31:0/2688295903 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0bd001e030 con 0x7f0bd4102f80 2024-04-25T13:10:47.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.233+0000 7f0bd9985700 1 --2- 172.21.15.31:0/2688295903 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0bc00396b0 0x7f0bc003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:47.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.234+0000 7f0bd9985700 1 --2- 172.21.15.31:0/2688295903 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0bc00396b0 0x7f0bc003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4006fd0 tx=0x7f0bc4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:47.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.238+0000 7f0bcb7fe700 1 -- 172.21.15.31:0/2688295903 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bd0013070 con 0x7f0bd4102f80 2024-04-25T13:10:47.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.453+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2688295903 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f0bd402e170 con 0x7f0bc00396b0 2024-04-25T13:10:47.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.456+0000 7f0bcb7fe700 1 -- 172.21.15.31:0/2688295903 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0bd402e170 con 0x7f0bc00396b0 2024-04-25T13:10:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.458+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2688295903 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0bc00396b0 msgr2=0x7f0bc003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.458+0000 7f0bdc3ea700 1 --2- 172.21.15.31:0/2688295903 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0bc00396b0 0x7f0bc003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4006fd0 tx=0x7f0bc4006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.458+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2688295903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd4102f80 msgr2=0x7f0bd4197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.458+0000 7f0bdc3ea700 1 --2- 172.21.15.31:0/2688295903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd4102f80 0x7f0bd4197a40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005b40 tx=0x7f0bd0004f80 comp rx=0 tx=0).stop 2024-04-25T13:10:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.459+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2688295903 shutdown_connections 2024-04-25T13:10:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.459+0000 7f0bdc3ea700 1 --2- 172.21.15.31:0/2688295903 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0bc00396b0 0x7f0bc003bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.459+0000 7f0bdc3ea700 1 --2- 172.21.15.31:0/2688295903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bd4102f80 0x7f0bd4197a40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.459+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2688295903 >> 172.21.15.31:0/2688295903 conn(0x7f0bd40fc950 msgr2=0x7f0bd40fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.459+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2688295903 shutdown_connections 2024-04-25T13:10:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:47.459+0000 7f0bdc3ea700 1 -- 172.21.15.31:0/2688295903 wait complete. 2024-04-25T13:10:48.244 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi066 2024-04-25T13:10:48.244 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-25T13:10:48.244 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/etc/ceph/ceph.conf 2024-04-25T13:10:48.263 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-25T13:10:48.263 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:10:48.322 INFO:tasks.cephadm:Adding host smithi066 to orchestrator... 2024-04-25T13:10:48.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch host add smithi066 2024-04-25T13:10:48.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:48 smithi031 ceph-mon[29457]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:48.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:48.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:48.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:10:48.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:10:48.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:48.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:48.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:10:49.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:49 smithi031 ceph-mon[29457]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T13:10:49.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:49 smithi031 ceph-mon[29457]: Updating smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:10:49.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:49 smithi031 ceph-mon[29457]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:10:49.629 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:49 smithi031 ceph-mon[29457]: Updating smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:10:50.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.112+0000 7f6bd0e48700 1 -- 172.21.15.31:0/3886361200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bcc103620 msgr2=0x7f6bcc103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:50.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.112+0000 7f6bd0e48700 1 --2- 172.21.15.31:0/3886361200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bcc103620 0x7f6bcc103a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009910 tx=0x7f6bbc009c20 comp rx=0 tx=0).stop 2024-04-25T13:10:50.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.112+0000 7f6bd0e48700 1 -- 172.21.15.31:0/3886361200 shutdown_connections 2024-04-25T13:10:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.112+0000 7f6bd0e48700 1 --2- 172.21.15.31:0/3886361200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bcc103620 0x7f6bcc103a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.112+0000 7f6bd0e48700 1 -- 172.21.15.31:0/3886361200 >> 172.21.15.31:0/3886361200 conn(0x7f6bcc0fce70 msgr2=0x7f6bcc0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.112+0000 7f6bd0e48700 1 -- 172.21.15.31:0/3886361200 shutdown_connections 2024-04-25T13:10:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.112+0000 7f6bd0e48700 1 -- 172.21.15.31:0/3886361200 wait complete. 2024-04-25T13:10:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.113+0000 7f6bd0e48700 1 Processor -- start 2024-04-25T13:10:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.113+0000 7f6bd0e48700 1 -- start start 2024-04-25T13:10:50.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.113+0000 7f6bd0e48700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bcc103620 0x7f6bcc197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.113+0000 7f6bd0e48700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc10c9c0 con 0x7f6bcc103620 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.113+0000 7f6bca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bcc103620 0x7f6bcc197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.113+0000 7f6bca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bcc103620 0x7f6bcc197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36456/0 (socket says 172.21.15.31:36456) 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.113+0000 7f6bca59c700 1 -- 172.21.15.31:0/1294053916 learned_addr learned my addr 172.21.15.31:0/1294053916 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.114+0000 7f6bca59c700 1 -- 172.21.15.31:0/1294053916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc198450 con 0x7f6bcc103620 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.114+0000 7f6bca59c700 1 --2- 172.21.15.31:0/1294053916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bcc103620 0x7f6bcc197f10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0056d0 tx=0x7f6bbc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.114+0000 7f6bbb7fe700 1 -- 172.21.15.31:0/1294053916 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bbc026070 con 0x7f6bcc103620 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.114+0000 7f6bbb7fe700 1 -- 172.21.15.31:0/1294053916 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bbc004370 con 0x7f6bcc103620 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.114+0000 7f6bd0e48700 1 -- 172.21.15.31:0/1294053916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc1986e0 con 0x7f6bcc103620 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.114+0000 7f6bbb7fe700 1 -- 172.21.15.31:0/1294053916 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bbc0039b0 con 0x7f6bcc103620 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.114+0000 7f6bd0e48700 1 -- 172.21.15.31:0/1294053916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc198b40 con 0x7f6bcc103620 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.115+0000 7f6bbb7fe700 1 -- 172.21.15.31:0/1294053916 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6bbc029070 con 0x7f6bcc103620 2024-04-25T13:10:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.115+0000 7f6bbb7fe700 1 --2- 172.21.15.31:0/1294053916 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6bb4039650 0x7f6bb403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.115+0000 7f6bbb7fe700 1 -- 172.21.15.31:0/1294053916 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6bbc01e070 con 0x7f6bcc103620 2024-04-25T13:10:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.115+0000 7f6bc9d9b700 1 --2- 172.21.15.31:0/1294053916 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6bb4039650 0x7f6bb403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.115+0000 7f6bd0e48700 1 -- 172.21.15.31:0/1294053916 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc06b1f0 con 0x7f6bcc103620 2024-04-25T13:10:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.116+0000 7f6bc9d9b700 1 --2- 172.21.15.31:0/1294053916 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6bb4039650 0x7f6bb403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0006fd0 tx=0x7f6bc0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:50.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.120+0000 7f6bbb7fe700 1 -- 172.21.15.31:0/1294053916 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bbc030020 con 0x7f6bcc103620 2024-04-25T13:10:50.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:50.336+0000 7f6bd0e48700 1 -- 172.21.15.31:0/1294053916 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi066", "target": ["mon-mgr", ""]}) -- 0x7f6bcc199270 con 0x7f6bb4039650 2024-04-25T13:10:50.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:50 smithi031 ceph-mon[29457]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi066", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:10:51.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:51 smithi031 ceph-mon[29457]: Deploying cephadm binary to smithi066 2024-04-25T13:10:52.821 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:52 smithi031 ceph-mon[29457]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:10:54.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:54.694+0000 7f6bbb7fe700 1 -- 172.21.15.31:0/1294053916 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6bcc199270 con 0x7f6bb4039650 2024-04-25T13:10:54.695 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi066' with addr '172.21.15.66' 2024-04-25T13:10:54.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:54.697+0000 7f6bd0e48700 1 -- 172.21.15.31:0/1294053916 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6bb4039650 msgr2=0x7f6bb403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:54.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:54.697+0000 7f6bd0e48700 1 --2- 172.21.15.31:0/1294053916 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6bb4039650 0x7f6bb403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0006fd0 tx=0x7f6bc0006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:54.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:54.697+0000 7f6bd0e48700 1 -- 172.21.15.31:0/1294053916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bcc103620 msgr2=0x7f6bcc197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:54.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:54.697+0000 7f6bd0e48700 1 --2- 172.21.15.31:0/1294053916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bcc103620 0x7f6bcc197f10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0056d0 tx=0x7f6bbc0057b0 comp rx=0 tx=0).stop 2024-04-25T13:10:54.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:54.698+0000 7f6bd0e48700 1 -- 172.21.15.31:0/1294053916 shutdown_connections 2024-04-25T13:10:54.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:54.698+0000 7f6bd0e48700 1 --2- 172.21.15.31:0/1294053916 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6bb4039650 0x7f6bb403bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:54.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:54.698+0000 7f6bd0e48700 1 --2- 172.21.15.31:0/1294053916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bcc103620 0x7f6bcc197f10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:54.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:54.698+0000 7f6bd0e48700 1 -- 172.21.15.31:0/1294053916 >> 172.21.15.31:0/1294053916 conn(0x7f6bcc0fce70 msgr2=0x7f6bcc1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:54.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:54.698+0000 7f6bd0e48700 1 -- 172.21.15.31:0/1294053916 shutdown_connections 2024-04-25T13:10:54.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:54.698+0000 7f6bd0e48700 1 -- 172.21.15.31:0/1294053916 wait complete. 2024-04-25T13:10:54.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:54 smithi031 ceph-mon[29457]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:10:54.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:54.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:10:55.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T13:10:55.612 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:10:56.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:56 smithi031 ceph-mon[29457]: Added host smithi066 2024-04-25T13:10:56.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:56 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:56.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:56 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:57.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.025+0000 7f822e376700 1 -- 172.21.15.31:0/519823597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8228108550 msgr2=0x7f8228108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:57.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.025+0000 7f822e376700 1 --2- 172.21.15.31:0/519823597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8228108550 0x7f8228108930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8218009910 tx=0x7f8218009c20 comp rx=0 tx=0).stop 2024-04-25T13:10:57.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.025+0000 7f822e376700 1 -- 172.21.15.31:0/519823597 shutdown_connections 2024-04-25T13:10:57.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.025+0000 7f822e376700 1 --2- 172.21.15.31:0/519823597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8228108550 0x7f8228108930 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:57.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.025+0000 7f822e376700 1 -- 172.21.15.31:0/519823597 >> 172.21.15.31:0/519823597 conn(0x7f8228077ac0 msgr2=0x7f8228077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:57.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.025+0000 7f822e376700 1 -- 172.21.15.31:0/519823597 shutdown_connections 2024-04-25T13:10:57.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.025+0000 7f822e376700 1 -- 172.21.15.31:0/519823597 wait complete. 2024-04-25T13:10:57.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.026+0000 7f822e376700 1 Processor -- start 2024-04-25T13:10:57.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.026+0000 7f822e376700 1 -- start start 2024-04-25T13:10:57.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.026+0000 7f822e376700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8228108550 0x7f8228101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:57.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.026+0000 7f822e376700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f822810c9b0 con 0x7f8228108550 2024-04-25T13:10:57.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.026+0000 7f8227fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8228108550 0x7f8228101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:57.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.026+0000 7f8227fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8228108550 0x7f8228101bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42690/0 (socket says 172.21.15.31:42690) 2024-04-25T13:10:57.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.026+0000 7f8227fff700 1 -- 172.21.15.31:0/1589483185 learned_addr learned my addr 172.21.15.31:0/1589483185 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:57.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.027+0000 7f8227fff700 1 -- 172.21.15.31:0/1589483185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8228103aa0 con 0x7f8228108550 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.027+0000 7f8227fff700 1 --2- 172.21.15.31:0/1589483185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8228108550 0x7f8228101bb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82180056d0 tx=0x7f82180057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.027+0000 7f82257fa700 1 -- 172.21.15.31:0/1589483185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8218026070 con 0x7f8228108550 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.027+0000 7f82257fa700 1 -- 172.21.15.31:0/1589483185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821800bc00 con 0x7f8228108550 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.027+0000 7f822e376700 1 -- 172.21.15.31:0/1589483185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8228102150 con 0x7f8228108550 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.027+0000 7f82257fa700 1 -- 172.21.15.31:0/1589483185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8218003920 con 0x7f8228108550 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.027+0000 7f822e376700 1 -- 172.21.15.31:0/1589483185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8228102610 con 0x7f8228108550 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.028+0000 7f82257fa700 1 -- 172.21.15.31:0/1589483185 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8218029070 con 0x7f8228108550 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.028+0000 7f82257fa700 1 --2- 172.21.15.31:0/1589483185 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8210039650 0x7f821003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.028+0000 7f82257fa700 1 -- 172.21.15.31:0/1589483185 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f821801e070 con 0x7f8228108550 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.029+0000 7f82277fe700 1 --2- 172.21.15.31:0/1589483185 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8210039650 0x7f821003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.029+0000 7f822e376700 1 -- 172.21.15.31:0/1589483185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82280640a0 con 0x7f8228108550 2024-04-25T13:10:57.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.029+0000 7f82277fe700 1 --2- 172.21.15.31:0/1589483185 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8210039650 0x7f821003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f821c006fd0 tx=0x7f821c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.034+0000 7f82257fa700 1 -- 172.21.15.31:0/1589483185 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8218013070 con 0x7f8228108550 2024-04-25T13:10:57.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.267+0000 7f822e376700 1 -- 172.21.15.31:0/1589483185 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82281028a0 con 0x7f8210039650 2024-04-25T13:10:57.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.269+0000 7f82257fa700 1 -- 172.21.15.31:0/1589483185 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f82281028a0 con 0x7f8210039650 2024-04-25T13:10:57.269 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:10:57.269 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}] 2024-04-25T13:10:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.271+0000 7f822e376700 1 -- 172.21.15.31:0/1589483185 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8210039650 msgr2=0x7f821003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.271+0000 7f822e376700 1 --2- 172.21.15.31:0/1589483185 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8210039650 0x7f821003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f821c006fd0 tx=0x7f821c006e40 comp rx=0 tx=0).stop 2024-04-25T13:10:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.271+0000 7f822e376700 1 -- 172.21.15.31:0/1589483185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8228108550 msgr2=0x7f8228101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.271+0000 7f822e376700 1 --2- 172.21.15.31:0/1589483185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8228108550 0x7f8228101bb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82180056d0 tx=0x7f82180057b0 comp rx=0 tx=0).stop 2024-04-25T13:10:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.271+0000 7f822e376700 1 -- 172.21.15.31:0/1589483185 shutdown_connections 2024-04-25T13:10:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.271+0000 7f822e376700 1 --2- 172.21.15.31:0/1589483185 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8210039650 0x7f821003bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.271+0000 7f822e376700 1 --2- 172.21.15.31:0/1589483185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8228108550 0x7f8228101bb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.271+0000 7f822e376700 1 -- 172.21.15.31:0/1589483185 >> 172.21.15.31:0/1589483185 conn(0x7f8228077ac0 msgr2=0x7f8228103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.271+0000 7f822e376700 1 -- 172.21.15.31:0/1589483185 shutdown_connections 2024-04-25T13:10:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:57.271+0000 7f822e376700 1 -- 172.21.15.31:0/1589483185 wait complete. 2024-04-25T13:10:57.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:57 smithi031 ceph-mon[29457]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:10:57.868 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi136 2024-04-25T13:10:57.868 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T13:10:57.868 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.conf 2024-04-25T13:10:57.888 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T13:10:57.888 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:10:57.944 INFO:tasks.cephadm:Adding host smithi136 to orchestrator... 2024-04-25T13:10:57.945 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch host add smithi136 2024-04-25T13:10:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:10:58.659 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:58 smithi031 ceph-mon[29457]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:10:58.660 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:58 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:10:59.457 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:10:59 smithi031 ceph-mon[29457]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:10:59.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.662+0000 7f2479e3d700 1 -- 172.21.15.31:0/700378649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24741015a0 msgr2=0x7f2474101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:10:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.662+0000 7f2479e3d700 1 --2- 172.21.15.31:0/700378649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24741015a0 0x7f2474101980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2468009910 tx=0x7f2468009c20 comp rx=0 tx=0).stop 2024-04-25T13:10:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.663+0000 7f2479e3d700 1 -- 172.21.15.31:0/700378649 shutdown_connections 2024-04-25T13:10:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.663+0000 7f2479e3d700 1 --2- 172.21.15.31:0/700378649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24741015a0 0x7f2474101980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:10:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.663+0000 7f2479e3d700 1 -- 172.21.15.31:0/700378649 >> 172.21.15.31:0/700378649 conn(0x7f24740fce50 msgr2=0x7f24740ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:10:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.663+0000 7f2479e3d700 1 -- 172.21.15.31:0/700378649 shutdown_connections 2024-04-25T13:10:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.663+0000 7f2479e3d700 1 -- 172.21.15.31:0/700378649 wait complete. 2024-04-25T13:10:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.663+0000 7f2479e3d700 1 Processor -- start 2024-04-25T13:10:59.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.663+0000 7f2479e3d700 1 -- start start 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.664+0000 7f2479e3d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24741015a0 0x7f2474196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.664+0000 7f2479e3d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f247410b820 con 0x7f24741015a0 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.664+0000 7f24737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24741015a0 0x7f2474196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.664+0000 7f24737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24741015a0 0x7f2474196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42704/0 (socket says 172.21.15.31:42704) 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.664+0000 7f24737fe700 1 -- 172.21.15.31:0/483090281 learned_addr learned my addr 172.21.15.31:0/483090281 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.664+0000 7f24737fe700 1 -- 172.21.15.31:0/483090281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247406b6d0 con 0x7f24741015a0 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.664+0000 7f24737fe700 1 --2- 172.21.15.31:0/483090281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24741015a0 0x7f2474196ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2468006010 tx=0x7f2468005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.664+0000 7f2470ff9700 1 -- 172.21.15.31:0/483090281 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2468026070 con 0x7f24741015a0 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.664+0000 7f2470ff9700 1 -- 172.21.15.31:0/483090281 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f246800bba0 con 0x7f24741015a0 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.664+0000 7f2479e3d700 1 -- 172.21.15.31:0/483090281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2474197010 con 0x7f24741015a0 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.665+0000 7f2470ff9700 1 -- 172.21.15.31:0/483090281 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2468003910 con 0x7f24741015a0 2024-04-25T13:10:59.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.665+0000 7f2479e3d700 1 -- 172.21.15.31:0/483090281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24741974d0 con 0x7f24741015a0 2024-04-25T13:10:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.665+0000 7f2470ff9700 1 -- 172.21.15.31:0/483090281 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2468029070 con 0x7f24741015a0 2024-04-25T13:10:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.665+0000 7f2470ff9700 1 --2- 172.21.15.31:0/483090281 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f245c0396b0 0x7f245c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:10:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.666+0000 7f2470ff9700 1 -- 172.21.15.31:0/483090281 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f246801e070 con 0x7f24741015a0 2024-04-25T13:10:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.666+0000 7f2472ffd700 1 --2- 172.21.15.31:0/483090281 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f245c0396b0 0x7f245c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:10:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.666+0000 7f2479e3d700 1 -- 172.21.15.31:0/483090281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2474102a80 con 0x7f24741015a0 2024-04-25T13:10:59.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.666+0000 7f2472ffd700 1 --2- 172.21.15.31:0/483090281 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f245c0396b0 0x7f245c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2464006fd0 tx=0x7f2464006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:10:59.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.671+0000 7f2470ff9700 1 -- 172.21.15.31:0/483090281 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2468030070 con 0x7f24741015a0 2024-04-25T13:10:59.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:10:59.885+0000 7f2479e3d700 1 -- 172.21.15.31:0/483090281 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}) -- 0x7f247406aec0 con 0x7f245c0396b0 2024-04-25T13:11:01.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:01 smithi031 ceph-mon[29457]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:01.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:01 smithi031 ceph-mon[29457]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:01.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:01.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:01.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:01.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:01.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:11:01.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:01.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:02.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:02 smithi031 ceph-mon[29457]: Deploying cephadm binary to smithi136 2024-04-25T13:11:02.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:02 smithi031 ceph-mon[29457]: Updating smithi066:/etc/ceph/ceph.conf 2024-04-25T13:11:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:02 smithi031 ceph-mon[29457]: Updating smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:03.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:03 smithi031 ceph-mon[29457]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:04.139 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi136' with addr '172.21.15.136' 2024-04-25T13:11:04.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:04.138+0000 7f2470ff9700 1 -- 172.21.15.31:0/483090281 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f247406aec0 con 0x7f245c0396b0 2024-04-25T13:11:04.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:04.141+0000 7f2479e3d700 1 -- 172.21.15.31:0/483090281 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f245c0396b0 msgr2=0x7f245c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:04.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:04.141+0000 7f2479e3d700 1 --2- 172.21.15.31:0/483090281 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f245c0396b0 0x7f245c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2464006fd0 tx=0x7f2464006e40 comp rx=0 tx=0).stop 2024-04-25T13:11:04.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:04.141+0000 7f2479e3d700 1 -- 172.21.15.31:0/483090281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24741015a0 msgr2=0x7f2474196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:04.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:04.141+0000 7f2479e3d700 1 --2- 172.21.15.31:0/483090281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24741015a0 0x7f2474196ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2468006010 tx=0x7f2468005780 comp rx=0 tx=0).stop 2024-04-25T13:11:04.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:04.141+0000 7f2479e3d700 1 -- 172.21.15.31:0/483090281 shutdown_connections 2024-04-25T13:11:04.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:04.141+0000 7f2479e3d700 1 --2- 172.21.15.31:0/483090281 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f245c0396b0 0x7f245c03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:04.141+0000 7f2479e3d700 1 --2- 172.21.15.31:0/483090281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24741015a0 0x7f2474196ad0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:04.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:04.141+0000 7f2479e3d700 1 -- 172.21.15.31:0/483090281 >> 172.21.15.31:0/483090281 conn(0x7f24740fce50 msgr2=0x7f2474102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:04.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:04.141+0000 7f2479e3d700 1 -- 172.21.15.31:0/483090281 shutdown_connections 2024-04-25T13:11:04.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:04.141+0000 7f2479e3d700 1 -- 172.21.15.31:0/483090281 wait complete. 2024-04-25T13:11:04.695 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:04 smithi031 ceph-mon[29457]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:11:04.695 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:04.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:04.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:04.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:04.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:04.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T13:11:04.973 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:05.719 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:05 smithi031 ceph-mon[29457]: Updating smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:11:05.720 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:05 smithi031 ceph-mon[29457]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:05.720 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:05 smithi031 ceph-mon[29457]: Added host smithi136 2024-04-25T13:11:05.720 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:05 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:05.720 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:05 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:06.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.524+0000 7fc3b2c29700 1 -- 172.21.15.31:0/546087052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3ac108550 msgr2=0x7fc3ac108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:06.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.524+0000 7fc3b2c29700 1 --2- 172.21.15.31:0/546087052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3ac108550 0x7fc3ac108930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009910 tx=0x7fc3a0009c20 comp rx=0 tx=0).stop 2024-04-25T13:11:06.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.524+0000 7fc3b2c29700 1 -- 172.21.15.31:0/546087052 shutdown_connections 2024-04-25T13:11:06.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.524+0000 7fc3b2c29700 1 --2- 172.21.15.31:0/546087052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3ac108550 0x7fc3ac108930 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.524+0000 7fc3b2c29700 1 -- 172.21.15.31:0/546087052 >> 172.21.15.31:0/546087052 conn(0x7fc3ac0fdf70 msgr2=0x7fc3ac100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:06.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.524+0000 7fc3b2c29700 1 -- 172.21.15.31:0/546087052 shutdown_connections 2024-04-25T13:11:06.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.524+0000 7fc3b2c29700 1 -- 172.21.15.31:0/546087052 wait complete. 2024-04-25T13:11:06.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.525+0000 7fc3b2c29700 1 Processor -- start 2024-04-25T13:11:06.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.525+0000 7fc3b2c29700 1 -- start start 2024-04-25T13:11:06.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.525+0000 7fc3b2c29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3ac108550 0x7fc3ac197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:06.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.525+0000 7fc3b2c29700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ac10c9c0 con 0x7fc3ac108550 2024-04-25T13:11:06.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.525+0000 7fc3b09c5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3ac108550 0x7fc3ac197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:06.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.525+0000 7fc3b09c5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3ac108550 0x7fc3ac197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59658/0 (socket says 172.21.15.31:59658) 2024-04-25T13:11:06.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.525+0000 7fc3b09c5700 1 -- 172.21.15.31:0/2303279228 learned_addr learned my addr 172.21.15.31:0/2303279228 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:11:06.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.526+0000 7fc3b09c5700 1 -- 172.21.15.31:0/2303279228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ac198400 con 0x7fc3ac108550 2024-04-25T13:11:06.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.526+0000 7fc3b09c5700 1 --2- 172.21.15.31:0/2303279228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3ac108550 0x7fc3ac197ec0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00056d0 tx=0x7fc3a00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:06.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.526+0000 7fc3a9ffb700 1 -- 172.21.15.31:0/2303279228 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3a0026070 con 0x7fc3ac108550 2024-04-25T13:11:06.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.526+0000 7fc3b2c29700 1 -- 172.21.15.31:0/2303279228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ac198690 con 0x7fc3ac108550 2024-04-25T13:11:06.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.526+0000 7fc3a9ffb700 1 -- 172.21.15.31:0/2303279228 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3a000bc00 con 0x7fc3ac108550 2024-04-25T13:11:06.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.526+0000 7fc3a9ffb700 1 -- 172.21.15.31:0/2303279228 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3a0003920 con 0x7fc3ac108550 2024-04-25T13:11:06.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.526+0000 7fc3b2c29700 1 -- 172.21.15.31:0/2303279228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ac198af0 con 0x7fc3ac108550 2024-04-25T13:11:06.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.527+0000 7fc3a9ffb700 1 -- 172.21.15.31:0/2303279228 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc3a0029070 con 0x7fc3ac108550 2024-04-25T13:11:06.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.527+0000 7fc3b2c29700 1 -- 172.21.15.31:0/2303279228 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ac103bc0 con 0x7fc3ac108550 2024-04-25T13:11:06.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.527+0000 7fc3a9ffb700 1 --2- 172.21.15.31:0/2303279228 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc39c039650 0x7fc39c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:06.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.527+0000 7fc3a9ffb700 1 -- 172.21.15.31:0/2303279228 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc3a001e070 con 0x7fc3ac108550 2024-04-25T13:11:06.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.527+0000 7fc3abfff700 1 --2- 172.21.15.31:0/2303279228 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc39c039650 0x7fc39c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:06.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.528+0000 7fc3abfff700 1 --2- 172.21.15.31:0/2303279228 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc39c039650 0x7fc39c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc398006fd0 tx=0x7fc398006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:06.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.532+0000 7fc3a9ffb700 1 -- 172.21.15.31:0/2303279228 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3a0013070 con 0x7fc3ac108550 2024-04-25T13:11:06.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.744+0000 7fc3b2c29700 1 -- 172.21.15.31:0/2303279228 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3ac199220 con 0x7fc39c039650 2024-04-25T13:11:06.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.745+0000 7fc3a9ffb700 1 -- 172.21.15.31:0/2303279228 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fc3ac199220 con 0x7fc39c039650 2024-04-25T13:11:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:11:06.746 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}] 2024-04-25T13:11:06.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.747+0000 7fc3b2c29700 1 -- 172.21.15.31:0/2303279228 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc39c039650 msgr2=0x7fc39c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:06.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.747+0000 7fc3b2c29700 1 --2- 172.21.15.31:0/2303279228 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc39c039650 0x7fc39c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc398006fd0 tx=0x7fc398006e40 comp rx=0 tx=0).stop 2024-04-25T13:11:06.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.747+0000 7fc3b2c29700 1 -- 172.21.15.31:0/2303279228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3ac108550 msgr2=0x7fc3ac197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:06.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.747+0000 7fc3b2c29700 1 --2- 172.21.15.31:0/2303279228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3ac108550 0x7fc3ac197ec0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00056d0 tx=0x7fc3a00057b0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.748+0000 7fc3b2c29700 1 -- 172.21.15.31:0/2303279228 shutdown_connections 2024-04-25T13:11:06.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.748+0000 7fc3b2c29700 1 --2- 172.21.15.31:0/2303279228 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc39c039650 0x7fc39c03bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.748+0000 7fc3b2c29700 1 --2- 172.21.15.31:0/2303279228 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3ac108550 0x7fc3ac197ec0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:06.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.748+0000 7fc3b2c29700 1 -- 172.21.15.31:0/2303279228 >> 172.21.15.31:0/2303279228 conn(0x7fc3ac0fdf70 msgr2=0x7fc3ac0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:06.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.748+0000 7fc3b2c29700 1 -- 172.21.15.31:0/2303279228 shutdown_connections 2024-04-25T13:11:06.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:06.748+0000 7fc3b2c29700 1 -- 172.21.15.31:0/2303279228 wait complete. 2024-04-25T13:11:07.368 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T13:11:07.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T13:11:07.634 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:07.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:07 smithi031 ceph-mon[29457]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:07.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:07 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:08.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:08 smithi031 ceph-mon[29457]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:11:09.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.145+0000 7fcf7f99e700 1 -- 172.21.15.31:0/277544424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf781084e0 msgr2=0x7fcf781088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.145+0000 7fcf7f99e700 1 --2- 172.21.15.31:0/277544424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf781084e0 0x7fcf781088c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009910 tx=0x7fcf74009c20 comp rx=0 tx=0).stop 2024-04-25T13:11:09.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.145+0000 7fcf7f99e700 1 -- 172.21.15.31:0/277544424 shutdown_connections 2024-04-25T13:11:09.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.145+0000 7fcf7f99e700 1 --2- 172.21.15.31:0/277544424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf781084e0 0x7fcf781088c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.145+0000 7fcf7f99e700 1 -- 172.21.15.31:0/277544424 >> 172.21.15.31:0/277544424 conn(0x7fcf780fdf80 msgr2=0x7fcf781003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:09.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.145+0000 7fcf7f99e700 1 -- 172.21.15.31:0/277544424 shutdown_connections 2024-04-25T13:11:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.145+0000 7fcf7f99e700 1 -- 172.21.15.31:0/277544424 wait complete. 2024-04-25T13:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.146+0000 7fcf7f99e700 1 Processor -- start 2024-04-25T13:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.146+0000 7fcf7f99e700 1 -- start start 2024-04-25T13:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.146+0000 7fcf7f99e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf781084e0 0x7fcf78074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.146+0000 7fcf7f99e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7810c940 con 0x7fcf781084e0 2024-04-25T13:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.146+0000 7fcf7d73a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf781084e0 0x7fcf78074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.146+0000 7fcf7d73a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf781084e0 0x7fcf78074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59670/0 (socket says 172.21.15.31:59670) 2024-04-25T13:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.146+0000 7fcf7d73a700 1 -- 172.21.15.31:0/1852217529 learned_addr learned my addr 172.21.15.31:0/1852217529 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.147+0000 7fcf7d73a700 1 -- 172.21.15.31:0/1852217529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf78076c30 con 0x7fcf781084e0 2024-04-25T13:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.147+0000 7fcf7d73a700 1 --2- 172.21.15.31:0/1852217529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf781084e0 0x7fcf78074d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf740056d0 tx=0x7fcf740057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.147+0000 7fcf6e7fc700 1 -- 172.21.15.31:0/1852217529 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf74026070 con 0x7fcf781084e0 2024-04-25T13:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.147+0000 7fcf6e7fc700 1 -- 172.21.15.31:0/1852217529 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf7400bc00 con 0x7fcf781084e0 2024-04-25T13:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.147+0000 7fcf7f99e700 1 -- 172.21.15.31:0/1852217529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf780752e0 con 0x7fcf781084e0 2024-04-25T13:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.147+0000 7fcf6e7fc700 1 -- 172.21.15.31:0/1852217529 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf74003920 con 0x7fcf781084e0 2024-04-25T13:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.147+0000 7fcf7f99e700 1 -- 172.21.15.31:0/1852217529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf780757a0 con 0x7fcf781084e0 2024-04-25T13:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.148+0000 7fcf7f99e700 1 -- 172.21.15.31:0/1852217529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf78103b90 con 0x7fcf781084e0 2024-04-25T13:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.148+0000 7fcf6e7fc700 1 -- 172.21.15.31:0/1852217529 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcf74029070 con 0x7fcf781084e0 2024-04-25T13:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.148+0000 7fcf6e7fc700 1 --2- 172.21.15.31:0/1852217529 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcf64039650 0x7fcf6403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.148+0000 7fcf6e7fc700 1 -- 172.21.15.31:0/1852217529 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcf7401e070 con 0x7fcf781084e0 2024-04-25T13:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.148+0000 7fcf7cf39700 1 --2- 172.21.15.31:0/1852217529 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcf64039650 0x7fcf6403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.149+0000 7fcf7cf39700 1 --2- 172.21.15.31:0/1852217529 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcf64039650 0x7fcf6403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006fd0 tx=0x7fcf68006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.153+0000 7fcf6e7fc700 1 -- 172.21.15.31:0/1852217529 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf74013070 con 0x7fcf781084e0 2024-04-25T13:11:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.364+0000 7fcf7f99e700 1 -- 172.21.15.31:0/1852217529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcf78076270 con 0x7fcf781084e0 2024-04-25T13:11:09.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.418+0000 7fcf6e7fc700 1 -- 172.21.15.31:0/1852217529 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fcf7400fe20 con 0x7fcf781084e0 2024-04-25T13:11:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.421+0000 7fcf7f99e700 1 -- 172.21.15.31:0/1852217529 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcf64039650 msgr2=0x7fcf6403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.421+0000 7fcf7f99e700 1 --2- 172.21.15.31:0/1852217529 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcf64039650 0x7fcf6403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006fd0 tx=0x7fcf68006e40 comp rx=0 tx=0).stop 2024-04-25T13:11:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.421+0000 7fcf7f99e700 1 -- 172.21.15.31:0/1852217529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf781084e0 msgr2=0x7fcf78074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:09.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.421+0000 7fcf7f99e700 1 --2- 172.21.15.31:0/1852217529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf781084e0 0x7fcf78074d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf740056d0 tx=0x7fcf740057b0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.421+0000 7fcf7f99e700 1 -- 172.21.15.31:0/1852217529 shutdown_connections 2024-04-25T13:11:09.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.421+0000 7fcf7f99e700 1 --2- 172.21.15.31:0/1852217529 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcf64039650 0x7fcf6403bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.421+0000 7fcf7f99e700 1 --2- 172.21.15.31:0/1852217529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf781084e0 0x7fcf78074d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:09.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.421+0000 7fcf7f99e700 1 -- 172.21.15.31:0/1852217529 >> 172.21.15.31:0/1852217529 conn(0x7fcf780fdf80 msgr2=0x7fcf7810a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:09.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.421+0000 7fcf7f99e700 1 -- 172.21.15.31:0/1852217529 shutdown_connections 2024-04-25T13:11:09.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:09.421+0000 7fcf7f99e700 1 -- 172.21.15.31:0/1852217529 wait complete. 2024-04-25T13:11:09.422 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-04-25T13:11:09.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:09 smithi031 ceph-mon[29457]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:09.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:09 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1852217529' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T13:11:10.019 INFO:tasks.cephadm:Adding mon.a on smithi031 2024-04-25T13:11:10.019 INFO:tasks.cephadm:Adding mon.b on smithi066 2024-04-25T13:11:10.019 INFO:tasks.cephadm:Adding mon.c on smithi136 2024-04-25T13:11:10.019 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi031:172.21.15.31=a;smithi066:172.21.15.66=b;smithi136:172.21.15.136=c' 2024-04-25T13:11:10.284 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T13:11:10.331 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T13:11:10.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:10 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1852217529' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T13:11:10.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:10 smithi031 ceph-mon[29457]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:11:10.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:10.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:10.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:10.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:10.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:11:10.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:10.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:11.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:11 smithi031 ceph-mon[29457]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:11.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:11 smithi031 ceph-mon[29457]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T13:11:11.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:11.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:11.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:11.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.858+0000 7f90c8321700 1 -- 172.21.15.136:0/4247116221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c01015f0 msgr2=0x7f90c01019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:11.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.858+0000 7f90c8321700 1 --2- 172.21.15.136:0/4247116221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c01015f0 0x7f90c01019d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009910 tx=0x7f90bc009c20 comp rx=0 tx=0).stop 2024-04-25T13:11:11.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.858+0000 7f90c8321700 1 -- 172.21.15.136:0/4247116221 shutdown_connections 2024-04-25T13:11:11.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.858+0000 7f90c8321700 1 --2- 172.21.15.136:0/4247116221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c01015f0 0x7f90c01019d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:11.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.858+0000 7f90c8321700 1 -- 172.21.15.136:0/4247116221 >> 172.21.15.136:0/4247116221 conn(0x7f90c00fcea0 msgr2=0x7f90c00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:11.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.858+0000 7f90c8321700 1 -- 172.21.15.136:0/4247116221 shutdown_connections 2024-04-25T13:11:11.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.858+0000 7f90c8321700 1 -- 172.21.15.136:0/4247116221 wait complete. 2024-04-25T13:11:11.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.859+0000 7f90c8321700 1 Processor -- start 2024-04-25T13:11:11.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.859+0000 7f90c8321700 1 -- start start 2024-04-25T13:11:11.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.859+0000 7f90c8321700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c01015f0 0x7f90c0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.859+0000 7f90c8321700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f90c010b870 con 0x7f90c01015f0 2024-04-25T13:11:11.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.859+0000 7f90c60bd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c01015f0 0x7f90c0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:11.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.859+0000 7f90c60bd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c01015f0 0x7f90c0198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.136:34608/0 (socket says 172.21.15.136:34608) 2024-04-25T13:11:11.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.859+0000 7f90c60bd700 1 -- 172.21.15.136:0/647769082 learned_addr learned my addr 172.21.15.136:0/647769082 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T13:11:11.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.859+0000 7f90c60bd700 1 -- 172.21.15.136:0/647769082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c0199290 con 0x7f90c01015f0 2024-04-25T13:11:11.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.860+0000 7f90c60bd700 1 --2- 172.21.15.136:0/647769082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c01015f0 0x7f90c0198d50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0056d0 tx=0x7f90bc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:11.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.860+0000 7f90b77fe700 1 -- 172.21.15.136:0/647769082 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90bc026070 con 0x7f90c01015f0 2024-04-25T13:11:11.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.860+0000 7f90c8321700 1 -- 172.21.15.136:0/647769082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c0199580 con 0x7f90c01015f0 2024-04-25T13:11:11.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.860+0000 7f90b77fe700 1 -- 172.21.15.136:0/647769082 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90bc00bba0 con 0x7f90c01015f0 2024-04-25T13:11:11.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.860+0000 7f90b77fe700 1 -- 172.21.15.136:0/647769082 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90bc003920 con 0x7f90c01015f0 2024-04-25T13:11:11.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.860+0000 7f90c8321700 1 -- 172.21.15.136:0/647769082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c0199a40 con 0x7f90c01015f0 2024-04-25T13:11:11.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.861+0000 7f90c8321700 1 -- 172.21.15.136:0/647769082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c0102ad0 con 0x7f90c01015f0 2024-04-25T13:11:11.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.861+0000 7f90b77fe700 1 -- 172.21.15.136:0/647769082 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f90bc029070 con 0x7f90c01015f0 2024-04-25T13:11:11.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.861+0000 7f90b77fe700 1 --2- 172.21.15.136:0/647769082 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90ac039650 0x7f90ac03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:11.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.861+0000 7f90b77fe700 1 -- 172.21.15.136:0/647769082 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f90bc01e070 con 0x7f90c01015f0 2024-04-25T13:11:11.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.861+0000 7f90c58bc700 1 --2- 172.21.15.136:0/647769082 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90ac039650 0x7f90ac03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:11.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.862+0000 7f90c58bc700 1 --2- 172.21.15.136:0/647769082 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90ac039650 0x7f90ac03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90b0006fd0 tx=0x7f90b0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:11.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:11.867+0000 7f90b77fe700 1 -- 172.21.15.136:0/647769082 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90bc013070 con 0x7f90c01015f0 2024-04-25T13:11:12.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.076+0000 7f90c8321700 1 -- 172.21.15.136:0/647769082 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi066:172.21.15.66=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}) -- 0x7f90c006aea0 con 0x7f90ac039650 2024-04-25T13:11:12.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.083+0000 7f90b77fe700 1 -- 172.21.15.136:0/647769082 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f90c006aea0 con 0x7f90ac039650 2024-04-25T13:11:12.083 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mon update... 2024-04-25T13:11:12.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.085+0000 7f90c8321700 1 -- 172.21.15.136:0/647769082 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90ac039650 msgr2=0x7f90ac03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:12.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.085+0000 7f90c8321700 1 --2- 172.21.15.136:0/647769082 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90ac039650 0x7f90ac03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90b0006fd0 tx=0x7f90b0006e40 comp rx=0 tx=0).stop 2024-04-25T13:11:12.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.085+0000 7f90c8321700 1 -- 172.21.15.136:0/647769082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c01015f0 msgr2=0x7f90c0198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:12.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.085+0000 7f90c8321700 1 --2- 172.21.15.136:0/647769082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c01015f0 0x7f90c0198d50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0056d0 tx=0x7f90bc0057b0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.085+0000 7f90c8321700 1 -- 172.21.15.136:0/647769082 shutdown_connections 2024-04-25T13:11:12.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.085+0000 7f90c8321700 1 --2- 172.21.15.136:0/647769082 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90ac039650 0x7f90ac03bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.085+0000 7f90c8321700 1 --2- 172.21.15.136:0/647769082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c01015f0 0x7f90c0198d50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:12.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.085+0000 7f90c8321700 1 -- 172.21.15.136:0/647769082 >> 172.21.15.136:0/647769082 conn(0x7f90c00fcea0 msgr2=0x7f90c0102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:12.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.085+0000 7f90c8321700 1 -- 172.21.15.136:0/647769082 shutdown_connections 2024-04-25T13:11:12.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:12.085+0000 7f90c8321700 1 -- 172.21.15.136:0/647769082 wait complete. 2024-04-25T13:11:12.674 DEBUG:teuthology.orchestra.run.smithi066:mon.b> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T13:11:12.677 DEBUG:teuthology.orchestra.run.smithi136:mon.c> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T13:11:12.679 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:11:12.679 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:11:12.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:12 smithi031 ceph-mon[29457]: Updating smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:12.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:12 smithi031 ceph-mon[29457]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:11:12.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:12 smithi031 ceph-mon[29457]: Updating smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:11:12.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:12.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:12.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:12.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:12.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:12.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:11:12.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:13.380 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:13.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:13 smithi031 ceph-mon[29457]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:13.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:13 smithi031 ceph-mon[29457]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi066:172.21.15.66=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:13.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:13 smithi031 ceph-mon[29457]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi066:172.21.15.66=b;smithi136:172.21.15.136=c;count:3 2024-04-25T13:11:13.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:13 smithi031 ceph-mon[29457]: Deploying daemon mon.c on smithi136 2024-04-25T13:11:14.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.887+0000 7fbeae0c8700 1 -- 172.21.15.136:0/3747538690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbea81015f0 msgr2=0x7fbea81019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:14.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.887+0000 7fbeae0c8700 1 --2- 172.21.15.136:0/3747538690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbea81015f0 0x7fbea81019d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009910 tx=0x7fbe9c009c20 comp rx=0 tx=0).stop 2024-04-25T13:11:14.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.887+0000 7fbeae0c8700 1 -- 172.21.15.136:0/3747538690 shutdown_connections 2024-04-25T13:11:14.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.887+0000 7fbeae0c8700 1 --2- 172.21.15.136:0/3747538690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbea81015f0 0x7fbea81019d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:14.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.887+0000 7fbeae0c8700 1 -- 172.21.15.136:0/3747538690 >> 172.21.15.136:0/3747538690 conn(0x7fbea8076ea0 msgr2=0x7fbea80772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:14.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.888+0000 7fbeae0c8700 1 -- 172.21.15.136:0/3747538690 shutdown_connections 2024-04-25T13:11:14.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.888+0000 7fbeae0c8700 1 -- 172.21.15.136:0/3747538690 wait complete. 2024-04-25T13:11:14.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.888+0000 7fbeae0c8700 1 Processor -- start 2024-04-25T13:11:14.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.888+0000 7fbeae0c8700 1 -- start start 2024-04-25T13:11:14.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.888+0000 7fbeae0c8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbea81015f0 0x7fbea8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:14.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.888+0000 7fbeae0c8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbea810b820 con 0x7fbea81015f0 2024-04-25T13:11:14.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.889+0000 7fbea77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbea81015f0 0x7fbea8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:14.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.889+0000 7fbea77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbea81015f0 0x7fbea8197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.136:34614/0 (socket says 172.21.15.136:34614) 2024-04-25T13:11:14.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.889+0000 7fbea77fe700 1 -- 172.21.15.136:0/587082209 learned_addr learned my addr 172.21.15.136:0/587082209 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T13:11:14.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.889+0000 7fbea77fe700 1 -- 172.21.15.136:0/587082209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea81983c0 con 0x7fbea81015f0 2024-04-25T13:11:14.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.889+0000 7fbea77fe700 1 --2- 172.21.15.136:0/587082209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbea81015f0 0x7fbea8197e80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0056d0 tx=0x7fbe9c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:14.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.889+0000 7fbea4ff9700 1 -- 172.21.15.136:0/587082209 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe9c026070 con 0x7fbea81015f0 2024-04-25T13:11:14.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.889+0000 7fbeae0c8700 1 -- 172.21.15.136:0/587082209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea81986b0 con 0x7fbea81015f0 2024-04-25T13:11:14.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.889+0000 7fbea4ff9700 1 -- 172.21.15.136:0/587082209 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe9c00bba0 con 0x7fbea81015f0 2024-04-25T13:11:14.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.889+0000 7fbea4ff9700 1 -- 172.21.15.136:0/587082209 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe9c003920 con 0x7fbea81015f0 2024-04-25T13:11:14.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.889+0000 7fbeae0c8700 1 -- 172.21.15.136:0/587082209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea819c750 con 0x7fbea81015f0 2024-04-25T13:11:14.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.890+0000 7fbeae0c8700 1 -- 172.21.15.136:0/587082209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea8102a80 con 0x7fbea81015f0 2024-04-25T13:11:14.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.891+0000 7fbea4ff9700 1 -- 172.21.15.136:0/587082209 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbe9c029070 con 0x7fbea81015f0 2024-04-25T13:11:14.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.891+0000 7fbea4ff9700 1 --2- 172.21.15.136:0/587082209 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fbe90039650 0x7fbe9003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:14.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.891+0000 7fbea4ff9700 1 -- 172.21.15.136:0/587082209 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbe9c01e070 con 0x7fbea81015f0 2024-04-25T13:11:14.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.891+0000 7fbea6ffd700 1 --2- 172.21.15.136:0/587082209 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fbe90039650 0x7fbe9003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:14.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.891+0000 7fbea6ffd700 1 --2- 172.21.15.136:0/587082209 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fbe90039650 0x7fbe9003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbe98006fd0 tx=0x7fbe98006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:14.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:14.896+0000 7fbea4ff9700 1 -- 172.21.15.136:0/587082209 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe9c013070 con 0x7fbea81015f0 2024-04-25T13:11:15.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.184+0000 7fbeae0c8700 1 -- 172.21.15.136:0/587082209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbea81991e0 con 0x7fbea81015f0 2024-04-25T13:11:15.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.185+0000 7fbea4ff9700 1 -- 172.21.15.136:0/587082209 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fbe9c0282c0 con 0x7fbea81015f0 2024-04-25T13:11:15.185 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:11:15.186 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","modified":"2024-04-25T13:09:31.556580Z","created":"2024-04-25T13:09:31.556580Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T13:11:15.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.187+0000 7fbeae0c8700 1 -- 172.21.15.136:0/587082209 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fbe90039650 msgr2=0x7fbe9003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:15.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.187+0000 7fbeae0c8700 1 --2- 172.21.15.136:0/587082209 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fbe90039650 0x7fbe9003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbe98006fd0 tx=0x7fbe98006e40 comp rx=0 tx=0).stop 2024-04-25T13:11:15.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.187+0000 7fbeae0c8700 1 -- 172.21.15.136:0/587082209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbea81015f0 msgr2=0x7fbea8197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:15.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.187+0000 7fbeae0c8700 1 --2- 172.21.15.136:0/587082209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbea81015f0 0x7fbea8197e80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0056d0 tx=0x7fbe9c0057b0 comp rx=0 tx=0).stop 2024-04-25T13:11:15.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.187+0000 7fbeae0c8700 1 -- 172.21.15.136:0/587082209 shutdown_connections 2024-04-25T13:11:15.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.187+0000 7fbeae0c8700 1 --2- 172.21.15.136:0/587082209 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fbe90039650 0x7fbe9003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:15.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.187+0000 7fbeae0c8700 1 --2- 172.21.15.136:0/587082209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbea81015f0 0x7fbea8197e80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:15.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.187+0000 7fbeae0c8700 1 -- 172.21.15.136:0/587082209 >> 172.21.15.136:0/587082209 conn(0x7fbea8076ea0 msgr2=0x7fbea8069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:15.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.187+0000 7fbeae0c8700 1 -- 172.21.15.136:0/587082209 shutdown_connections 2024-04-25T13:11:15.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:15.187+0000 7fbeae0c8700 1 -- 172.21.15.136:0/587082209 wait complete. 2024-04-25T13:11:15.189 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-25T13:11:15.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:15 smithi031 ceph-mon[29457]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:15.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:15 smithi031 ceph-mon[29457]: from='client.? 172.21.15.136:0/587082209' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:11:16.778 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:11:16.778 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:11:17.056 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:11:17.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:17 smithi031 ceph-mon[29457]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:18.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.798+0000 7f072446a700 1 -- 172.21.15.136:0/905051742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f071c100eb0 msgr2=0x7f071c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:18.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.798+0000 7f072446a700 1 --2- 172.21.15.136:0/905051742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f071c100eb0 0x7f071c101290 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f070c009910 tx=0x7f070c009c20 comp rx=0 tx=0).stop 2024-04-25T13:11:18.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.798+0000 7f072446a700 1 -- 172.21.15.136:0/905051742 shutdown_connections 2024-04-25T13:11:18.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.798+0000 7f072446a700 1 --2- 172.21.15.136:0/905051742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f071c100eb0 0x7f071c101290 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:18.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.798+0000 7f072446a700 1 -- 172.21.15.136:0/905051742 >> 172.21.15.136:0/905051742 conn(0x7f071c0fcb50 msgr2=0x7f071c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:18.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.798+0000 7f072446a700 1 -- 172.21.15.136:0/905051742 shutdown_connections 2024-04-25T13:11:18.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.798+0000 7f072446a700 1 -- 172.21.15.136:0/905051742 wait complete. 2024-04-25T13:11:18.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.799+0000 7f072446a700 1 Processor -- start 2024-04-25T13:11:18.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.799+0000 7f072446a700 1 -- start start 2024-04-25T13:11:18.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.799+0000 7f072446a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f071c100eb0 0x7f071c0fd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:18.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.799+0000 7f072446a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f071c10b0e0 con 0x7f071c100eb0 2024-04-25T13:11:18.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.799+0000 7f0722206700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f071c100eb0 0x7f071c0fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:18.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.799+0000 7f0722206700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f071c100eb0 0x7f071c0fd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.136:48930/0 (socket says 172.21.15.136:48930) 2024-04-25T13:11:18.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.799+0000 7f0722206700 1 -- 172.21.15.136:0/4236424644 learned_addr learned my addr 172.21.15.136:0/4236424644 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T13:11:18.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.800+0000 7f0722206700 1 -- 172.21.15.136:0/4236424644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c0feef0 con 0x7f071c100eb0 2024-04-25T13:11:18.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.800+0000 7f0722206700 1 --2- 172.21.15.136:0/4236424644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f071c100eb0 0x7f071c0fd000 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f070c006010 tx=0x7f070c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:18.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.800+0000 7f07137fe700 1 -- 172.21.15.136:0/4236424644 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f070c026070 con 0x7f071c100eb0 2024-04-25T13:11:18.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.800+0000 7f072446a700 1 -- 172.21.15.136:0/4236424644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071c0fd5a0 con 0x7f071c100eb0 2024-04-25T13:11:18.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.800+0000 7f07137fe700 1 -- 172.21.15.136:0/4236424644 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f070c00bc00 con 0x7f071c100eb0 2024-04-25T13:11:18.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.800+0000 7f072446a700 1 -- 172.21.15.136:0/4236424644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071c0fda60 con 0x7f071c100eb0 2024-04-25T13:11:18.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.800+0000 7f07137fe700 1 -- 172.21.15.136:0/4236424644 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f070c0038d0 con 0x7f071c100eb0 2024-04-25T13:11:18.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.801+0000 7f072446a700 1 -- 172.21.15.136:0/4236424644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f071c102340 con 0x7f071c100eb0 2024-04-25T13:11:18.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.801+0000 7f07137fe700 1 -- 172.21.15.136:0/4236424644 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f070c029070 con 0x7f071c100eb0 2024-04-25T13:11:18.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.801+0000 7f07137fe700 1 --2- 172.21.15.136:0/4236424644 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0708039650 0x7f070803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:18.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.801+0000 7f07137fe700 1 -- 172.21.15.136:0/4236424644 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f070c01e070 con 0x7f071c100eb0 2024-04-25T13:11:18.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.801+0000 7f0721a05700 1 --2- 172.21.15.136:0/4236424644 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0708039650 0x7f070803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:18.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.802+0000 7f0721a05700 1 --2- 172.21.15.136:0/4236424644 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0708039650 0x7f070803bb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0718006fd0 tx=0x7f0718006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:18.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:18.807+0000 7f07137fe700 1 -- 172.21.15.136:0/4236424644 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f070c013070 con 0x7f071c100eb0 2024-04-25T13:11:19.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.094+0000 7f072446a700 1 -- 172.21.15.136:0/4236424644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f071c0fe530 con 0x7f071c100eb0 2024-04-25T13:11:19.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.095+0000 7f07137fe700 1 -- 172.21.15.136:0/4236424644 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f070c04cf10 con 0x7f071c100eb0 2024-04-25T13:11:19.095 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:11:19.095 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","modified":"2024-04-25T13:09:31.556580Z","created":"2024-04-25T13:09:31.556580Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T13:11:19.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.097+0000 7f072446a700 1 -- 172.21.15.136:0/4236424644 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0708039650 msgr2=0x7f070803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:19.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.097+0000 7f072446a700 1 --2- 172.21.15.136:0/4236424644 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0708039650 0x7f070803bb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0718006fd0 tx=0x7f0718006e40 comp rx=0 tx=0).stop 2024-04-25T13:11:19.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.097+0000 7f072446a700 1 -- 172.21.15.136:0/4236424644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f071c100eb0 msgr2=0x7f071c0fd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:19.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.097+0000 7f072446a700 1 --2- 172.21.15.136:0/4236424644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f071c100eb0 0x7f071c0fd000 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f070c006010 tx=0x7f070c005780 comp rx=0 tx=0).stop 2024-04-25T13:11:19.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.097+0000 7f072446a700 1 -- 172.21.15.136:0/4236424644 shutdown_connections 2024-04-25T13:11:19.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.097+0000 7f072446a700 1 --2- 172.21.15.136:0/4236424644 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0708039650 0x7f070803bb10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:19.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.097+0000 7f072446a700 1 --2- 172.21.15.136:0/4236424644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f071c100eb0 0x7f071c0fd000 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:19.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.097+0000 7f072446a700 1 -- 172.21.15.136:0/4236424644 >> 172.21.15.136:0/4236424644 conn(0x7f071c0fcb50 msgr2=0x7f071c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:19.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.097+0000 7f072446a700 1 -- 172.21.15.136:0/4236424644 shutdown_connections 2024-04-25T13:11:19.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:19.097+0000 7f072446a700 1 -- 172.21.15.136:0/4236424644 wait complete. 2024-04-25T13:11:19.098 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-25T13:11:20.843 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:11:20.843 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:11:21.088 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:11:24.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:24 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:24.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:11:24.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: mon.a calling monitor election 2024-04-25T13:11:24.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:11:24.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:24.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:11:24.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:11:24.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: mon.c calling monitor election 2024-04-25T13:11:24.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: monmap epoch 2 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: last_changed 2024-04-25T13:11:19.111216+0000 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: created 2024-04-25T13:09:31.556580+0000 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: min_mon_release 19 (squid) 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: election_strategy: 1 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: fsmap 2024-04-25T13:11:24.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:11:24.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: mgrmap e13: x(active, since 52s) 2024-04-25T13:11:24.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: overall HEALTH_OK 2024-04-25T13:11:24.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:24.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:24.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:24.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:11:24.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:25.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:25 smithi136 ceph-mon[33038]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:25.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:25 smithi136 ceph-mon[33038]: Deploying daemon mon.b on smithi066 2024-04-25T13:11:25.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:25 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:11:25.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:25 smithi031 ceph-mon[29457]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:25.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:25 smithi031 ceph-mon[29457]: Deploying daemon mon.b on smithi066 2024-04-25T13:11:25.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:25 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:11:25.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.627+0000 7f0ea8a83700 1 -- 172.21.15.136:0/1213574785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e8c0057d0 msgr2=0x7f0e8c007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:25.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.627+0000 7f0ea8a83700 1 --2- 172.21.15.136:0/1213574785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e8c0057d0 0x7f0e8c007c60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4101380 tx=0x7f0e9800feb0 comp rx=0 tx=0).stop 2024-04-25T13:11:25.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.627+0000 7f0ea8a83700 1 -- 172.21.15.136:0/1213574785 shutdown_connections 2024-04-25T13:11:25.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.627+0000 7f0ea8a83700 1 --2- 172.21.15.136:0/1213574785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e8c0057d0 0x7f0e8c007c60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:25.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.627+0000 7f0ea8a83700 1 -- 172.21.15.136:0/1213574785 >> 172.21.15.136:0/1213574785 conn(0x7f0ea40fcf80 msgr2=0x7f0ea40ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:25.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.627+0000 7f0ea8a83700 1 -- 172.21.15.136:0/1213574785 shutdown_connections 2024-04-25T13:11:25.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.627+0000 7f0ea8a83700 1 -- 172.21.15.136:0/1213574785 wait complete. 2024-04-25T13:11:25.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.629+0000 7f0ea8a83700 1 Processor -- start 2024-04-25T13:11:25.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.629+0000 7f0ea8a83700 1 -- start start 2024-04-25T13:11:25.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.629+0000 7f0ea8a83700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e8c0057d0 0x7f0ea4195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:25.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.629+0000 7f0ea8a83700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea4196340 0x7f0ea419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:25.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.629+0000 7f0ea8a83700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41082f0 con 0x7f0ea4196340 2024-04-25T13:11:25.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.629+0000 7f0ea8a83700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea406bb80 con 0x7f0e8c0057d0 2024-04-25T13:11:25.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.629+0000 7f0ea259c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e8c0057d0 0x7f0ea4195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:25.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.629+0000 7f0ea1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea4196340 0x7f0ea419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:25.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.629+0000 7f0ea259c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e8c0057d0 0x7f0ea4195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:49806/0 (socket says 172.21.15.136:49806) 2024-04-25T13:11:25.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.629+0000 7f0ea259c700 1 -- 172.21.15.136:0/3871079977 learned_addr learned my addr 172.21.15.136:0/3871079977 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T13:11:25.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0ea259c700 1 -- 172.21.15.136:0/3871079977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e8c0057d0 msgr2=0x7f0ea4195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T13:11:25.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0ea259c700 1 -- 172.21.15.136:0/3871079977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e8c0057d0 msgr2=0x7f0ea4195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:11:25.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0ea259c700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e8c0057d0 0x7f0ea4195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:11:25.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0ea259c700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e8c0057d0 0x7f0ea4195e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:11:25.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0ea1d9b700 1 -- 172.21.15.136:0/3871079977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e8c0057d0 msgr2=0x7f0ea4195e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:11:25.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0ea1d9b700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e8c0057d0 0x7f0ea4195e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:25.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0ea1d9b700 1 -- 172.21.15.136:0/3871079977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea4196980 con 0x7f0ea4196340 2024-04-25T13:11:25.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0ea1d9b700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea4196340 0x7f0ea419a680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400b4d0 tx=0x7f0e9400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:25.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0e937fe700 1 -- 172.21.15.136:0/3871079977 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0e94011630 con 0x7f0ea4196340 2024-04-25T13:11:25.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0e937fe700 1 -- 172.21.15.136:0/3871079977 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e94011c80 con 0x7f0ea4196340 2024-04-25T13:11:25.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0ea8a83700 1 -- 172.21.15.136:0/3871079977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea419abc0 con 0x7f0ea4196340 2024-04-25T13:11:25.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0e937fe700 1 -- 172.21.15.136:0/3871079977 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0e94010da0 con 0x7f0ea4196340 2024-04-25T13:11:25.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.630+0000 7f0ea8a83700 1 -- 172.21.15.136:0/3871079977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea419b120 con 0x7f0ea4196340 2024-04-25T13:11:25.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.631+0000 7f0e937fe700 1 -- 172.21.15.136:0/3871079977 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0e94018020 con 0x7f0ea4196340 2024-04-25T13:11:25.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.632+0000 7f0e937fe700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e84039440 0x7f0e8403b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:25.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.632+0000 7f0e937fe700 1 -- 172.21.15.136:0/3871079977 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0e9404c1b0 con 0x7f0ea4196340 2024-04-25T13:11:25.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.632+0000 7f0ea8a83700 1 -- 172.21.15.136:0/3871079977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea4102b90 con 0x7f0ea4196340 2024-04-25T13:11:25.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.632+0000 7f0ea259c700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e84039440 0x7f0e8403b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:25.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.632+0000 7f0ea259c700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e84039440 0x7f0e8403b900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800fa60 tx=0x7f0e98005a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:25.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.638+0000 7f0e937fe700 1 -- 172.21.15.136:0/3871079977 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e9401f020 con 0x7f0ea4196340 2024-04-25T13:11:25.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.936+0000 7f0ea8a83700 1 -- 172.21.15.136:0/3871079977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0ea4197110 con 0x7f0ea4196340 2024-04-25T13:11:25.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.937+0000 7f0e937fe700 1 -- 172.21.15.136:0/3871079977 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f0e94018570 con 0x7f0ea4196340 2024-04-25T13:11:25.938 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:11:25.938 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":2,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","modified":"2024-04-25T13:11:19.111216Z","created":"2024-04-25T13:09:31.556580Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T13:11:25.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.939+0000 7f0ea8a83700 1 -- 172.21.15.136:0/3871079977 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e84039440 msgr2=0x7f0e8403b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:25.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.939+0000 7f0ea8a83700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e84039440 0x7f0e8403b900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800fa60 tx=0x7f0e98005a90 comp rx=0 tx=0).stop 2024-04-25T13:11:25.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.939+0000 7f0ea8a83700 1 -- 172.21.15.136:0/3871079977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea4196340 msgr2=0x7f0ea419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:25.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.939+0000 7f0ea8a83700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea4196340 0x7f0ea419a680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400b4d0 tx=0x7f0e9400b890 comp rx=0 tx=0).stop 2024-04-25T13:11:25.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.939+0000 7f0ea8a83700 1 -- 172.21.15.136:0/3871079977 shutdown_connections 2024-04-25T13:11:25.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.939+0000 7f0ea8a83700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e84039440 0x7f0e8403b900 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:25.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.939+0000 7f0ea8a83700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e8c0057d0 0x7f0ea4195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:25.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.939+0000 7f0ea8a83700 1 --2- 172.21.15.136:0/3871079977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ea4196340 0x7f0ea419a680 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:25.941 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.939+0000 7f0ea8a83700 1 -- 172.21.15.136:0/3871079977 >> 172.21.15.136:0/3871079977 conn(0x7f0ea40fcf80 msgr2=0x7f0ea40feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:25.941 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.939+0000 7f0ea8a83700 1 -- 172.21.15.136:0/3871079977 shutdown_connections 2024-04-25T13:11:25.941 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:25.939+0000 7f0ea8a83700 1 -- 172.21.15.136:0/3871079977 wait complete. 2024-04-25T13:11:25.941 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 2 2024-04-25T13:11:26.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:26 smithi136 ceph-mon[33038]: from='client.? 172.21.15.136:0/3871079977' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:11:26.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:26 smithi031 ceph-mon[29457]: from='client.? 172.21.15.136:0/3871079977' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:11:27.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:27 smithi136 ceph-mon[33038]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:27.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:27 smithi031 ceph-mon[29457]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:27.512 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:11:27.512 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:11:27.762 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:11:28.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:28 smithi066 systemd[1]: Started Ceph mon.b for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:11:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:11:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: mon.c calling monitor election 2024-04-25T13:11:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: mon.a calling monitor election 2024-04-25T13:11:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: monmap epoch 3 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: last_changed 2024-04-25T13:11:28.048656+0000 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: created 2024-04-25T13:09:31.556580+0000 2024-04-25T13:11:33.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: min_mon_release 19 (squid) 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: election_strategy: 1 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: fsmap 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: mgrmap e13: x(active, since 61s) 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: overall HEALTH_OK 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:33.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:11:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: mon.c calling monitor election 2024-04-25T13:11:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: mon.a calling monitor election 2024-04-25T13:11:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:11:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: monmap epoch 3 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: last_changed 2024-04-25T13:11:28.048656+0000 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: created 2024-04-25T13:09:31.556580+0000 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: min_mon_release 19 (squid) 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: election_strategy: 1 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: fsmap 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:11:33.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: mgrmap e13: x(active, since 61s) 2024-04-25T13:11:33.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: overall HEALTH_OK 2024-04-25T13:11:33.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:33.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:33.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:33.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:33.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.674+0000 7f9a4698c700 1 -- 172.21.15.136:0/274467946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a280035c0 msgr2=0x7f9a28005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:33.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.674+0000 7f9a4698c700 1 --2- 172.21.15.136:0/274467946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a280035c0 0x7f9a28005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9a40101380 tx=0x7f9a3000fe60 comp rx=0 tx=0).stop 2024-04-25T13:11:33.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.675+0000 7f9a4698c700 1 -- 172.21.15.136:0/274467946 shutdown_connections 2024-04-25T13:11:33.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.675+0000 7f9a4698c700 1 --2- 172.21.15.136:0/274467946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a280035c0 0x7f9a28005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:33.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.675+0000 7f9a4698c700 1 -- 172.21.15.136:0/274467946 >> 172.21.15.136:0/274467946 conn(0x7f9a400fcf80 msgr2=0x7f9a400ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:33.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.675+0000 7f9a4698c700 1 -- 172.21.15.136:0/274467946 shutdown_connections 2024-04-25T13:11:33.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.675+0000 7f9a4698c700 1 -- 172.21.15.136:0/274467946 wait complete. 2024-04-25T13:11:33.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.676+0000 7f9a4698c700 1 Processor -- start 2024-04-25T13:11:33.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.676+0000 7f9a4698c700 1 -- start start 2024-04-25T13:11:33.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a4698c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a280035c0 0x7f9a40196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:33.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a4698c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a40197520 0x7f9a4019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:33.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a4698c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a4019bdf0 0x7f9a4019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:33.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a4698c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9a40109360 con 0x7f9a4019bdf0 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a4698c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9a401091e0 con 0x7f9a280035c0 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a4698c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9a401094e0 con 0x7f9a40197520 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3f7fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a40197520 0x7f9a4019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3f7fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a40197520 0x7f9a4019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:33764/0 (socket says 172.21.15.136:33764) 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3f7fe700 1 -- 172.21.15.136:0/410307828 learned_addr learned my addr 172.21.15.136:0/410307828 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3ffff700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a280035c0 0x7f9a40196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3f7fe700 1 -- 172.21.15.136:0/410307828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a40197520 msgr2=0x7f9a4019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3f7fe700 1 -- 172.21.15.136:0/410307828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a40197520 msgr2=0x7f9a4019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3f7fe700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a40197520 0x7f9a4019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3f7fe700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a40197520 0x7f9a4019b8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a44f29700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a4019bdf0 0x7f9a4019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:33.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3ffff700 1 -- 172.21.15.136:0/410307828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a280035c0 msgr2=0x7f9a40196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3ffff700 1 -- 172.21.15.136:0/410307828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a280035c0 msgr2=0x7f9a40196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3ffff700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a280035c0 0x7f9a40196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.677+0000 7f9a3ffff700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a280035c0 0x7f9a40196fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.678+0000 7f9a44f29700 1 -- 172.21.15.136:0/410307828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a40197520 msgr2=0x7f9a4019b8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.678+0000 7f9a44f29700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a40197520 0x7f9a4019b8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.678+0000 7f9a44f29700 1 -- 172.21.15.136:0/410307828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a280035c0 msgr2=0x7f9a40196fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.678+0000 7f9a44f29700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a280035c0 0x7f9a40196fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.678+0000 7f9a44f29700 1 -- 172.21.15.136:0/410307828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a401a3b60 con 0x7f9a4019bdf0 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.678+0000 7f9a44f29700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a4019bdf0 0x7f9a4019c2a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800bd50 tx=0x7f9a3800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.678+0000 7f9a3d7fa700 1 -- 172.21.15.136:0/410307828 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38019070 con 0x7f9a4019bdf0 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.678+0000 7f9a4698c700 1 -- 172.21.15.136:0/410307828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4006acb0 con 0x7f9a4019bdf0 2024-04-25T13:11:33.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.678+0000 7f9a3d7fa700 1 -- 172.21.15.136:0/410307828 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a3800cd10 con 0x7f9a4019bdf0 2024-04-25T13:11:33.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.678+0000 7f9a3d7fa700 1 -- 172.21.15.136:0/410307828 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38012740 con 0x7f9a4019bdf0 2024-04-25T13:11:33.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.678+0000 7f9a4698c700 1 -- 172.21.15.136:0/410307828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4006b1f0 con 0x7f9a4019bdf0 2024-04-25T13:11:33.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.679+0000 7f9a4698c700 1 -- 172.21.15.136:0/410307828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a400640a0 con 0x7f9a4019bdf0 2024-04-25T13:11:33.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.679+0000 7f9a3d7fa700 1 -- 172.21.15.136:0/410307828 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9a3801c070 con 0x7f9a4019bdf0 2024-04-25T13:11:33.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.680+0000 7f9a3d7fa700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9a2c039ae0 0x7f9a2c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:33.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.680+0000 7f9a3d7fa700 1 -- 172.21.15.136:0/410307828 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9a3804c120 con 0x7f9a4019bdf0 2024-04-25T13:11:33.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.680+0000 7f9a3ffff700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9a2c039ae0 0x7f9a2c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:33.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.680+0000 7f9a3ffff700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9a2c039ae0 0x7f9a2c03bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000fa10 tx=0x7f9a30005ad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:33.685 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.685+0000 7f9a3d7fa700 1 -- 172.21.15.136:0/410307828 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a38007290 con 0x7f9a4019bdf0 2024-04-25T13:11:33.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.985+0000 7f9a4698c700 1 -- 172.21.15.136:0/410307828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9a40190530 con 0x7f9a4019bdf0 2024-04-25T13:11:33.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.986+0000 7f9a3d7fa700 1 -- 172.21.15.136:0/410307828 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f9a3800e090 con 0x7f9a4019bdf0 2024-04-25T13:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T13:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":3,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","modified":"2024-04-25T13:11:28.048656Z","created":"2024-04-25T13:09:31.556580Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T13:11:33.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.988+0000 7f9a4698c700 1 -- 172.21.15.136:0/410307828 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9a2c039ae0 msgr2=0x7f9a2c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:33.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.988+0000 7f9a4698c700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9a2c039ae0 0x7f9a2c03bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000fa10 tx=0x7f9a30005ad0 comp rx=0 tx=0).stop 2024-04-25T13:11:33.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.988+0000 7f9a4698c700 1 -- 172.21.15.136:0/410307828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a4019bdf0 msgr2=0x7f9a4019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:33.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.988+0000 7f9a4698c700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a4019bdf0 0x7f9a4019c2a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800bd50 tx=0x7f9a3800be30 comp rx=0 tx=0).stop 2024-04-25T13:11:33.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.988+0000 7f9a4698c700 1 -- 172.21.15.136:0/410307828 shutdown_connections 2024-04-25T13:11:33.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.988+0000 7f9a4698c700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a4019bdf0 0x7f9a4019c2a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:33.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.988+0000 7f9a4698c700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9a280035c0 0x7f9a40196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:33.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.988+0000 7f9a4698c700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9a2c039ae0 0x7f9a2c03bfa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:33.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.988+0000 7f9a4698c700 1 --2- 172.21.15.136:0/410307828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a40197520 0x7f9a4019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:33.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.988+0000 7f9a4698c700 1 -- 172.21.15.136:0/410307828 >> 172.21.15.136:0/410307828 conn(0x7f9a400fcf80 msgr2=0x7f9a40105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:33.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.989+0000 7f9a4698c700 1 -- 172.21.15.136:0/410307828 shutdown_connections 2024-04-25T13:11:33.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:33.989+0000 7f9a4698c700 1 -- 172.21.15.136:0/410307828 wait complete. 2024-04-25T13:11:33.990 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 3 2024-04-25T13:11:34.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:34 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:34.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:34 smithi136 ceph-mon[33038]: from='client.? 172.21.15.136:0/410307828' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:11:34.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:34 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:34.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:34 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:34.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:34 smithi031 ceph-mon[29457]: from='client.? 172.21.15.136:0/410307828' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:11:34.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:34 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:34.835 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T13:11:34.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T13:11:35.093 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:11:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: mon.c calling monitor election 2024-04-25T13:11:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: mon.a calling monitor election 2024-04-25T13:11:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: monmap epoch 3 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: last_changed 2024-04-25T13:11:28.048656+0000 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: created 2024-04-25T13:09:31.556580+0000 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: min_mon_release 19 (squid) 2024-04-25T13:11:35.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: election_strategy: 1 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: fsmap 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: mgrmap e13: x(active, since 61s) 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: overall HEALTH_OK 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='client.? 172.21.15.136:0/410307828' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:11:35.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: mon.b calling monitor election 2024-04-25T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: mon.b calling monitor election 2024-04-25T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: mon.a calling monitor election 2024-04-25T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: mon.c calling monitor election 2024-04-25T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: monmap epoch 3 2024-04-25T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: last_changed 2024-04-25T13:11:28.048656+0000 2024-04-25T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: created 2024-04-25T13:09:31.556580+0000 2024-04-25T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: min_mon_release 19 (squid) 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: election_strategy: 1 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: fsmap 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: mgrmap e13: x(active, since 63s) 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: overall HEALTH_OK 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T13:11:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: Updating smithi066:/etc/ceph/ceph.conf 2024-04-25T13:11:35.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T13:11:35.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: Updating smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:35.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: Updating smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:35.736 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:35 smithi031 ceph-mon[29457]: Updating smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:35.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: mon.b calling monitor election 2024-04-25T13:11:35.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: mon.b calling monitor election 2024-04-25T13:11:35.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: mon.a calling monitor election 2024-04-25T13:11:35.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: mon.c calling monitor election 2024-04-25T13:11:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:11:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: monmap epoch 3 2024-04-25T13:11:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:11:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: last_changed 2024-04-25T13:11:28.048656+0000 2024-04-25T13:11:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: created 2024-04-25T13:09:31.556580+0000 2024-04-25T13:11:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: min_mon_release 19 (squid) 2024-04-25T13:11:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: election_strategy: 1 2024-04-25T13:11:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T13:11:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T13:11:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-04-25T13:11:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: fsmap 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: mgrmap e13: x(active, since 63s) 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: overall HEALTH_OK 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: Updating smithi066:/etc/ceph/ceph.conf 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: Updating smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: Updating smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:35.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:35 smithi066 ceph-mon[33137]: Updating smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:35.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: mon.b calling monitor election 2024-04-25T13:11:35.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: mon.b calling monitor election 2024-04-25T13:11:35.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: mon.a calling monitor election 2024-04-25T13:11:35.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: mon.c calling monitor election 2024-04-25T13:11:35.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:11:35.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: monmap epoch 3 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: last_changed 2024-04-25T13:11:28.048656+0000 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: created 2024-04-25T13:09:31.556580+0000 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: min_mon_release 19 (squid) 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: election_strategy: 1 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: 2: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: fsmap 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: mgrmap e13: x(active, since 63s) 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: overall HEALTH_OK 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:35.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:35.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:35.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:35.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T13:11:35.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: Updating smithi066:/etc/ceph/ceph.conf 2024-04-25T13:11:35.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T13:11:35.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: Updating smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:35.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: Updating smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:35.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:35 smithi136 ceph-mon[33038]: Updating smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:11:36.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.692+0000 7f9de1372700 1 -- 172.21.15.31:0/3474309881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ddc1084e0 msgr2=0x7f9ddc1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.692+0000 7f9de1372700 1 --2- 172.21.15.31:0/3474309881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ddc1084e0 0x7f9ddc1088c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009910 tx=0x7f9dd4009c20 comp rx=0 tx=0).stop 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.692+0000 7f9de1372700 1 -- 172.21.15.31:0/3474309881 shutdown_connections 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.692+0000 7f9de1372700 1 --2- 172.21.15.31:0/3474309881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ddc1084e0 0x7f9ddc1088c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.692+0000 7f9de1372700 1 -- 172.21.15.31:0/3474309881 >> 172.21.15.31:0/3474309881 conn(0x7f9ddc0fdf80 msgr2=0x7f9ddc1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.692+0000 7f9de1372700 1 -- 172.21.15.31:0/3474309881 shutdown_connections 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.692+0000 7f9de1372700 1 -- 172.21.15.31:0/3474309881 wait complete. 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.692+0000 7f9de1372700 1 Processor -- start 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9de1372700 1 -- start start 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9de1372700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ddc1084e0 0x7f9ddc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9de1372700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ddc075280 0x7f9ddc075700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9de1372700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ddc1a42c0 0x7f9ddc1a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:36.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9de1372700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc10c940 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9de1372700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc10c7c0 con 0x7f9ddc1084e0 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9de1372700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc10cac0 con 0x7f9ddc075280 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9ddb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ddc1a42c0 0x7f9ddc1a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9ddb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ddc1a42c0 0x7f9ddc1a66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37230/0 (socket says 172.21.15.31:37230) 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9ddb7fe700 1 -- 172.21.15.31:0/4184817272 learned_addr learned my addr 172.21.15.31:0/4184817272 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9dda7fc700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ddc075280 0x7f9ddc075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9ddaffd700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ddc1084e0 0x7f9ddc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9ddb7fe700 1 -- 172.21.15.31:0/4184817272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ddc075280 msgr2=0x7f9ddc075700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9ddb7fe700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ddc075280 0x7f9ddc075700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9ddb7fe700 1 -- 172.21.15.31:0/4184817272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ddc1084e0 msgr2=0x7f9ddc074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9ddb7fe700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ddc1084e0 0x7f9ddc074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9ddb7fe700 1 -- 172.21.15.31:0/4184817272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc1a6de0 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9dda7fc700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ddc075280 0x7f9ddc075700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.693+0000 7f9ddb7fe700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ddc1a42c0 0x7f9ddc1a66b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00ba50 tx=0x7f9dcc00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.694+0000 7f9dd3fff700 1 -- 172.21.15.31:0/4184817272 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc019070 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.694+0000 7f9dd3fff700 1 -- 172.21.15.31:0/4184817272 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dcc014440 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.694+0000 7f9dd3fff700 1 -- 172.21.15.31:0/4184817272 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc007590 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.694+0000 7f9de1372700 1 -- 172.21.15.31:0/4184817272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc1a70d0 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.694+0000 7f9de1372700 1 -- 172.21.15.31:0/4184817272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc1a75c0 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.695+0000 7f9dd3fff700 1 -- 172.21.15.31:0/4184817272 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9dcc01c070 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.695+0000 7f9de1372700 1 -- 172.21.15.31:0/4184817272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ddc0640a0 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.695+0000 7f9dd3fff700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dc8039ae0 0x7f9dc803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.695+0000 7f9dd3fff700 1 -- 172.21.15.31:0/4184817272 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9dcc04bfb0 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.695+0000 7f9ddaffd700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dc8039ae0 0x7f9dc803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.695+0000 7f9ddaffd700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dc8039ae0 0x7f9dc803bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009750 tx=0x7f9dd4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.700+0000 7f9dd3fff700 1 -- 172.21.15.31:0/4184817272 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9dcc014910 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:11:36.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:11:36.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:11:36.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:36.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: Reconfiguring daemon mon.a on smithi031 2024-04-25T13:11:36.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:36.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.915+0000 7f9de1372700 1 -- 172.21.15.31:0/4184817272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9ddc1a7920 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.915+0000 7f9dd3fff700 1 -- 172.21.15.31:0/4184817272 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f9dcc01d3b0 con 0x7f9ddc1a42c0 2024-04-25T13:11:36.915 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:11:36.915 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-04-25T13:11:36.915 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:11:36.915 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-25T13:11:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.917+0000 7f9de1372700 1 -- 172.21.15.31:0/4184817272 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dc8039ae0 msgr2=0x7f9dc803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.917+0000 7f9de1372700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dc8039ae0 0x7f9dc803bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009750 tx=0x7f9dd4005fb0 comp rx=0 tx=0).stop 2024-04-25T13:11:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.917+0000 7f9de1372700 1 -- 172.21.15.31:0/4184817272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ddc1a42c0 msgr2=0x7f9ddc1a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.917+0000 7f9de1372700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ddc1a42c0 0x7f9ddc1a66b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00ba50 tx=0x7f9dcc00be10 comp rx=0 tx=0).stop 2024-04-25T13:11:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.917+0000 7f9de1372700 1 -- 172.21.15.31:0/4184817272 shutdown_connections 2024-04-25T13:11:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.917+0000 7f9de1372700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ddc1a42c0 0x7f9ddc1a66b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.917+0000 7f9de1372700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ddc1084e0 0x7f9ddc074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.917+0000 7f9de1372700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9dc8039ae0 0x7f9dc803bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.917+0000 7f9de1372700 1 --2- 172.21.15.31:0/4184817272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ddc075280 0x7f9ddc075700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.917+0000 7f9de1372700 1 -- 172.21.15.31:0/4184817272 >> 172.21.15.31:0/4184817272 conn(0x7f9ddc0fdf80 msgr2=0x7f9ddc103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.918+0000 7f9de1372700 1 -- 172.21.15.31:0/4184817272 shutdown_connections 2024-04-25T13:11:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:36.918+0000 7f9de1372700 1 -- 172.21.15.31:0/4184817272 wait complete. 2024-04-25T13:11:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:11:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:11:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:11:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: Reconfiguring daemon mon.a on smithi031 2024-04-25T13:11:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:11:37.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:11:37.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:11:37.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:37.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: Reconfiguring daemon mon.a on smithi031 2024-04-25T13:11:37.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:11:37.962 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:37.962 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:37 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/4184817272' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:37.990 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T13:11:37.991 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:11:37.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:11:38.018 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:11:38.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:11:38.083 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-25T13:11:38.083 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:11:38.110 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:38.110 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:37 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/4184817272' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:38.116 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-25T13:11:38.116 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:11:38.184 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T13:11:38.185 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:11:38.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:38.211 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:37 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/4184817272' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:38.216 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T13:11:38.216 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:11:38.285 INFO:tasks.cephadm:Adding mgr.x on smithi031 2024-04-25T13:11:38.331 INFO:tasks.cephadm:Adding mgr.y on smithi066 2024-04-25T13:11:38.331 INFO:tasks.cephadm:Adding mgr.z on smithi136 2024-04-25T13:11:38.331 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi031=x;smithi066=y;smithi136=z' 2024-04-25T13:11:38.572 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:11:39.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:39 smithi066 ceph-mon[33137]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:39.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:39.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:39.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:11:39.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:11:39.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:39.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:39 smithi136 ceph-mon[33038]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:39.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:39.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:39.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:11:39.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:11:39.667 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:39.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:39 smithi031 ceph-mon[29457]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:39.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:39.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:39.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:11:39.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:11:39.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:39.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.988+0000 7f51da27d700 1 -- 172.21.15.136:0/674478880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d41025c0 msgr2=0x7f51d41029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.988+0000 7f51da27d700 1 --2- 172.21.15.136:0/674478880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d41025c0 0x7f51d41029e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009910 tx=0x7f51c8009c20 comp rx=0 tx=0).stop 2024-04-25T13:11:39.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.988+0000 7f51da27d700 1 -- 172.21.15.136:0/674478880 shutdown_connections 2024-04-25T13:11:39.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.988+0000 7f51da27d700 1 --2- 172.21.15.136:0/674478880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d41025c0 0x7f51d41029e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.988+0000 7f51da27d700 1 -- 172.21.15.136:0/674478880 >> 172.21.15.136:0/674478880 conn(0x7f51d40fdb60 msgr2=0x7f51d40fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:39.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.988+0000 7f51da27d700 1 -- 172.21.15.136:0/674478880 shutdown_connections 2024-04-25T13:11:39.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.988+0000 7f51da27d700 1 -- 172.21.15.136:0/674478880 wait complete. 2024-04-25T13:11:39.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51da27d700 1 Processor -- start 2024-04-25T13:11:39.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51da27d700 1 -- start start 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51da27d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51d41025c0 0x7f51d40767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51da27d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51d4074e30 0x7f51d40752b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51da27d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d40757f0 0x7f51d41a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51da27d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4108500 con 0x7f51d40757f0 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51da27d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4108380 con 0x7f51d41025c0 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51da27d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4108680 con 0x7f51d4074e30 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51d37fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51d4074e30 0x7f51d40752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51d37fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51d4074e30 0x7f51d40752b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:51160/0 (socket says 172.21.15.136:51160) 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51d37fe700 1 -- 172.21.15.136:0/1461448171 learned_addr learned my addr 172.21.15.136:0/1461448171 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.989+0000 7f51d881a700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d40757f0 0x7f51d41a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51d3fff700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51d41025c0 0x7f51d40767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51d37fe700 1 -- 172.21.15.136:0/1461448171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51d41025c0 msgr2=0x7f51d40767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51d37fe700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51d41025c0 0x7f51d40767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51d37fe700 1 -- 172.21.15.136:0/1461448171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d40757f0 msgr2=0x7f51d41a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:39.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51d37fe700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d40757f0 0x7f51d41a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:39.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51d37fe700 1 -- 172.21.15.136:0/1461448171 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d41a2350 con 0x7f51d4074e30 2024-04-25T13:11:39.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51d37fe700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51d4074e30 0x7f51d40752b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f51c400b780 tx=0x7f51c400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:39.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51d17fa700 1 -- 172.21.15.136:0/1461448171 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4004230 con 0x7f51d4074e30 2024-04-25T13:11:39.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51d17fa700 1 -- 172.21.15.136:0/1461448171 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51c40043d0 con 0x7f51d4074e30 2024-04-25T13:11:39.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51da27d700 1 -- 172.21.15.136:0/1461448171 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d41a2640 con 0x7f51d4074e30 2024-04-25T13:11:39.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51d17fa700 1 -- 172.21.15.136:0/1461448171 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4011b20 con 0x7f51d4074e30 2024-04-25T13:11:39.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.990+0000 7f51da27d700 1 -- 172.21.15.136:0/1461448171 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d41a2c00 con 0x7f51d4074e30 2024-04-25T13:11:39.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.991+0000 7f51d17fa700 1 -- 172.21.15.136:0/1461448171 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f51c4011cc0 con 0x7f51d4074e30 2024-04-25T13:11:39.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.991+0000 7f51d17fa700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f51bc039ae0 0x7f51bc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:39.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.991+0000 7f51d17fa700 1 -- 172.21.15.136:0/1461448171 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f51c404d340 con 0x7f51d4074e30 2024-04-25T13:11:39.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.991+0000 7f51da27d700 1 -- 172.21.15.136:0/1461448171 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d40640a0 con 0x7f51d4074e30 2024-04-25T13:11:39.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.991+0000 7f51d3fff700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f51bc039ae0 0x7f51bc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:39.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.992+0000 7f51d3fff700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f51bc039ae0 0x7f51bc03bfa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f51c80050f0 tx=0x7f51c801a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:39.996 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:39.997+0000 7f51d17fa700 1 -- 172.21.15.136:0/1461448171 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51c4020500 con 0x7f51d4074e30 2024-04-25T13:11:40.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.216+0000 7f51da27d700 1 -- 172.21.15.136:0/1461448171 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi066=y;smithi136=z", "target": ["mon-mgr", ""]}) -- 0x7f51d4062e80 con 0x7f51bc039ae0 2024-04-25T13:11:40.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.222+0000 7f51d17fa700 1 -- 172.21.15.136:0/1461448171 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f51d4062e80 con 0x7f51bc039ae0 2024-04-25T13:11:40.222 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mgr update... 2024-04-25T13:11:40.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.224+0000 7f51da27d700 1 -- 172.21.15.136:0/1461448171 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f51bc039ae0 msgr2=0x7f51bc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:40.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.224+0000 7f51da27d700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f51bc039ae0 0x7f51bc03bfa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f51c80050f0 tx=0x7f51c801a040 comp rx=0 tx=0).stop 2024-04-25T13:11:40.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.224+0000 7f51da27d700 1 -- 172.21.15.136:0/1461448171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51d4074e30 msgr2=0x7f51d40752b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:40.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.224+0000 7f51da27d700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51d4074e30 0x7f51d40752b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f51c400b780 tx=0x7f51c400ba90 comp rx=0 tx=0).stop 2024-04-25T13:11:40.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.224+0000 7f51da27d700 1 -- 172.21.15.136:0/1461448171 shutdown_connections 2024-04-25T13:11:40.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.224+0000 7f51da27d700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d40757f0 0x7f51d41a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.224+0000 7f51da27d700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51d41025c0 0x7f51d40767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.224+0000 7f51da27d700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f51bc039ae0 0x7f51bc03bfa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.224+0000 7f51da27d700 1 --2- 172.21.15.136:0/1461448171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51d4074e30 0x7f51d40752b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:40.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.224+0000 7f51da27d700 1 -- 172.21.15.136:0/1461448171 >> 172.21.15.136:0/1461448171 conn(0x7f51d40fdb60 msgr2=0x7f51d40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:40.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.225+0000 7f51da27d700 1 -- 172.21.15.136:0/1461448171 shutdown_connections 2024-04-25T13:11:40.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:11:40.225+0000 7f51da27d700 1 -- 172.21.15.136:0/1461448171 wait complete. 2024-04-25T13:11:40.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:40 smithi136 ceph-mon[33038]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:11:40.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:40 smithi136 ceph-mon[33038]: Reconfiguring daemon mon.b on smithi066 2024-04-25T13:11:40.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:40.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:40.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:11:40.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:11:40.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:40.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:40.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:40 smithi031 ceph-mon[29457]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:11:40.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:40 smithi031 ceph-mon[29457]: Reconfiguring daemon mon.b on smithi066 2024-04-25T13:11:40.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:40.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:40.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:11:40.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:11:40.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:40.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:40.873 DEBUG:teuthology.orchestra.run.smithi066:mgr.y> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T13:11:40.876 DEBUG:teuthology.orchestra.run.smithi136:mgr.z> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T13:11:40.881 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T13:11:40.882 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:11:40.882 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:11:40.902 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:11:40.902 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-04-25T13:11:40.902 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:40 smithi066 ceph-mon[33137]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:11:40.902 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:40 smithi066 ceph-mon[33137]: Reconfiguring daemon mon.b on smithi066 2024-04-25T13:11:40.903 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:40.903 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:40.903 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:11:40.903 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:11:40.903 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:40.903 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:11:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:40.961 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T13:11:40.961 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:40.961 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:40.961 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 13:10:44.252192841 +0000 2024-04-25T13:11:40.961 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 13:05:41.296596399 +0000 2024-04-25T13:11:40.961 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 13:05:41.296596399 +0000 2024-04-25T13:11:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 13:05:41.296596399 +0000 2024-04-25T13:11:40.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:11:41.028 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T13:11:41.028 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T13:11:41.029 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000218683 s, 2.3 MB/s 2024-04-25T13:11:41.030 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:11:41.089 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-04-25T13:11:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:11:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:41.147 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T13:11:41.147 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:41.147 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:41.147 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 13:10:44.288192280 +0000 2024-04-25T13:11:41.147 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 13:05:41.652590609 +0000 2024-04-25T13:11:41.147 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 13:05:41.652590609 +0000 2024-04-25T13:11:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 13:05:41.652590609 +0000 2024-04-25T13:11:41.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:11:41.213 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T13:11:41.213 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T13:11:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000231734 s, 2.2 MB/s 2024-04-25T13:11:41.215 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:11:41.273 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-04-25T13:11:41.330 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:11:41.330 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:41.330 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T13:11:41.331 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:41.331 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:41.331 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 13:10:44.320191781 +0000 2024-04-25T13:11:41.331 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 13:05:41.998584980 +0000 2024-04-25T13:11:41.331 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 13:05:41.998584980 +0000 2024-04-25T13:11:41.331 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 13:05:41.998584980 +0000 2024-04-25T13:11:41.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:11:41.396 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T13:11:41.396 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T13:11:41.396 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000185714 s, 2.8 MB/s 2024-04-25T13:11:41.397 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:11:41.455 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-04-25T13:11:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:11:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:41.512 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T13:11:41.512 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:41.512 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:41.512 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-25 13:10:44.352191282 +0000 2024-04-25T13:11:41.512 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-25 13:05:42.345579337 +0000 2024-04-25T13:11:41.512 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-25 13:05:42.345579337 +0000 2024-04-25T13:11:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-25 13:05:42.345579337 +0000 2024-04-25T13:11:41.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:11:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-25T13:11:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-25T13:11:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.00020037 s, 2.6 MB/s 2024-04-25T13:11:41.578 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:11:41.635 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-04-25T13:11:41.635 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:11:41.655 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:11:41.655 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-04-25T13:11:41.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:41 smithi136 ceph-mon[33038]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:41.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:41 smithi136 ceph-mon[33038]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:11:41.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:41 smithi136 ceph-mon[33038]: Reconfiguring daemon mon.c on smithi136 2024-04-25T13:11:41.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:41 smithi136 ceph-mon[33038]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi066=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:41.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:41 smithi136 ceph-mon[33038]: Saving service mgr spec with placement smithi031=x;smithi066=y;smithi136=z;count:3 2024-04-25T13:11:41.714 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:11:41.714 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:41.715 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T13:11:41.715 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:41.715 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:41.715 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-25 13:11:27.478034078 +0000 2024-04-25T13:11:41.715 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-25 13:06:12.419352357 +0000 2024-04-25T13:11:41.715 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-25 13:06:12.419352357 +0000 2024-04-25T13:11:41.715 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-25 13:06:12.419352357 +0000 2024-04-25T13:11:41.715 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:11:41.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:41 smithi031 ceph-mon[29457]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:41.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:41 smithi031 ceph-mon[29457]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:11:41.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:41 smithi031 ceph-mon[29457]: Reconfiguring daemon mon.c on smithi136 2024-04-25T13:11:41.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:41 smithi031 ceph-mon[29457]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi066=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:41.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:41 smithi031 ceph-mon[29457]: Saving service mgr spec with placement smithi031=x;smithi066=y;smithi136=z;count:3 2024-04-25T13:11:41.779 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:41 smithi066 ceph-mon[33137]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:41.780 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:41 smithi066 ceph-mon[33137]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:11:41.780 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:41 smithi066 ceph-mon[33137]: Reconfiguring daemon mon.c on smithi136 2024-04-25T13:11:41.780 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:41 smithi066 ceph-mon[33137]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi066=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:41.780 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:41 smithi066 ceph-mon[33137]: Saving service mgr spec with placement smithi031=x;smithi066=y;smithi136=z;count:3 2024-04-25T13:11:41.782 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-25T13:11:41.782 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-25T13:11:41.782 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000226527 s, 2.3 MB/s 2024-04-25T13:11:41.783 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:11:41.842 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-04-25T13:11:41.903 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:11:41.903 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:41.903 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T13:11:41.903 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:41.903 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:41.903 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-25 13:11:27.478034078 +0000 2024-04-25T13:11:41.903 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-25 13:06:12.768350365 +0000 2024-04-25T13:11:41.903 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-25 13:06:12.768350365 +0000 2024-04-25T13:11:41.903 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-25 13:06:12.768350365 +0000 2024-04-25T13:11:41.903 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:11:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-25T13:11:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-25T13:11:41.968 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000266663 s, 1.9 MB/s 2024-04-25T13:11:41.969 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:11:42.027 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-04-25T13:11:42.084 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:11:42.084 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:42.084 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T13:11:42.084 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:42.085 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:42.085 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-25 13:11:27.479034073 +0000 2024-04-25T13:11:42.085 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-25 13:06:13.118348368 +0000 2024-04-25T13:11:42.085 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-25 13:06:13.118348368 +0000 2024-04-25T13:11:42.085 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-25 13:06:13.118348368 +0000 2024-04-25T13:11:42.085 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:11:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-25T13:11:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-25T13:11:42.149 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000187579 s, 2.7 MB/s 2024-04-25T13:11:42.150 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:11:42.207 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-04-25T13:11:42.263 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:11:42.264 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:42.264 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T13:11:42.264 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:42.264 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:42.264 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-04-25 13:11:27.479034073 +0000 2024-04-25T13:11:42.264 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-04-25 13:06:13.459346422 +0000 2024-04-25T13:11:42.264 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-04-25 13:06:13.459346422 +0000 2024-04-25T13:11:42.265 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-04-25 13:06:13.459346422 +0000 2024-04-25T13:11:42.265 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:11:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-04-25T13:11:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-04-25T13:11:42.329 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000174707 s, 2.9 MB/s 2024-04-25T13:11:42.329 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:11:42.386 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T13:11:42.386 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:11:42.402 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:11:42.402 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-04-25T13:11:42.458 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:11:42.458 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:42.458 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T13:11:42.458 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:42.458 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:42.458 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-25 13:11:17.830142834 +0000 2024-04-25T13:11:42.458 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-25 13:06:03.042286874 +0000 2024-04-25T13:11:42.458 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-25 13:06:03.042286874 +0000 2024-04-25T13:11:42.458 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-25 13:06:03.042286874 +0000 2024-04-25T13:11:42.459 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:11:42.522 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-25T13:11:42.522 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-25T13:11:42.522 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000221165 s, 2.3 MB/s 2024-04-25T13:11:42.523 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:11:42.579 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-04-25T13:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T13:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-25 13:11:17.831142812 +0000 2024-04-25T13:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-25 13:06:03.376278853 +0000 2024-04-25T13:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-25 13:06:03.376278853 +0000 2024-04-25T13:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-25 13:06:03.376278853 +0000 2024-04-25T13:11:42.636 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:11:42.698 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-25T13:11:42.698 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-25T13:11:42.698 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000261619 s, 2.0 MB/s 2024-04-25T13:11:42.699 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:11:42.756 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-04-25T13:11:42.811 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T13:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-25 13:11:17.831142812 +0000 2024-04-25T13:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-25 13:06:03.706270928 +0000 2024-04-25T13:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-25 13:06:03.706270928 +0000 2024-04-25T13:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-25 13:06:03.706270928 +0000 2024-04-25T13:11:42.812 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:11:42.872 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:42.872 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:42.872 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:42.872 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:42.873 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:42.873 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:42.873 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:42.873 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:11:42.873 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:11:42.873 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:11:42.873 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:42.873 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:42 smithi136 ceph-mon[33038]: Deploying daemon mgr.z on smithi136 2024-04-25T13:11:42.874 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-25T13:11:42.874 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-25T13:11:42.874 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000188562 s, 2.7 MB/s 2024-04-25T13:11:42.876 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:11:42.931 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-04-25T13:11:42.986 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:11:42.986 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:11:42.987 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T13:11:42.987 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:11:42.987 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:11:42.987 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-25 13:11:17.831142812 +0000 2024-04-25T13:11:42.987 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-25 13:06:04.042262858 +0000 2024-04-25T13:11:42.987 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-25 13:06:04.042262858 +0000 2024-04-25T13:11:42.987 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-25 13:06:04.042262858 +0000 2024-04-25T13:11:42.987 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:11:43.048 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-25T13:11:43.049 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-25T13:11:43.049 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000288096 s, 1.8 MB/s 2024-04-25T13:11:43.050 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:11:43.105 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2024-04-25T13:11:43.105 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:11:43.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:43.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:43.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:43.130 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:43.131 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:43.131 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:43.131 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:43.131 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:11:43.134 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:11:43.134 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:11:43.134 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:43.134 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:42 smithi031 ceph-mon[29457]: Deploying daemon mgr.z on smithi136 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:42 smithi066 ceph-mon[33137]: Deploying daemon mgr.z on smithi136 2024-04-25T13:11:43.353 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:44.638 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:44 smithi136 systemd[1]: Starting Ceph mgr.z for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:11:44.939 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:44 smithi136 ceph-mon[33038]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:44.940 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:44 smithi136 podman[34900]: 2024-04-25 13:11:44.638071263 +0000 UTC m=+0.113145376 container create 2d8062f9d8e8f6a3bc004d9e4974bee78fd677e383dcf3a0ed5a817b8b86f752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-25T13:11:44.940 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:44 smithi136 podman[34900]: 2024-04-25 13:11:44.542917733 +0000 UTC m=+0.017991846 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:11:44.940 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:44 smithi136 podman[34900]: 2024-04-25 13:11:44.857092605 +0000 UTC m=+0.332166720 container init 2d8062f9d8e8f6a3bc004d9e4974bee78fd677e383dcf3a0ed5a817b8b86f752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:11:44.940 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:44 smithi136 podman[34900]: 2024-04-25 13:11:44.859653263 +0000 UTC m=+0.334727378 container start 2d8062f9d8e8f6a3bc004d9e4974bee78fd677e383dcf3a0ed5a817b8b86f752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:11:44.940 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:44 smithi136 ceph-mgr[34920]: -- 172.21.15.136:0/2408403817 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5626f14405a0 con 0x5626f1414800 2024-04-25T13:11:44.940 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:44 smithi136 bash[34900]: 2d8062f9d8e8f6a3bc004d9e4974bee78fd677e383dcf3a0ed5a817b8b86f752 2024-04-25T13:11:44.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:44 smithi031 ceph-mon[29457]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:45.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:44 smithi066 ceph-mon[33137]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:45.197 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:44 smithi136 systemd[1]: Started Ceph mgr.z for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:11:45.197 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:45.012+0000 7fcff3f85200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:11:45.197 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:45.196+0000 7fcff3f85200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:11:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:11:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:11:45.293 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T13:11:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T13:11:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0134845 s, 778 MB/s 2024-04-25T13:11:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T13:11:45.468 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:45.298+0000 7fcff3f85200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:11:45.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2024-04-25T13:11:45.968 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:45.605+0000 7fcff3f85200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:11:45.968 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:45.689+0000 7fcff3f85200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:11:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:11:46.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:46.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:46.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:46.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:11:46.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:11:46.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:11:46.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:46.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:46 smithi031 ceph-mon[29457]: Deploying daemon mgr.y on smithi066 2024-04-25T13:11:46.307 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:46.307 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:46.307 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:46.307 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:11:46.307 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:11:46.307 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:11:46.307 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:46.307 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:46 smithi136 ceph-mon[33038]: Deploying daemon mgr.y on smithi066 2024-04-25T13:11:46.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:46.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:46.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:46.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:11:46.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:11:46.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:11:46.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:46.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:46 smithi066 ceph-mon[33137]: Deploying daemon mgr.y on smithi066 2024-04-25T13:11:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:47 smithi136 ceph-mon[33038]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:47.219 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:46 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:46.940+0000 7fcff3f85200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:11:47.219 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:47 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:47.114+0000 7fcff3f85200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:11:47.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:47 smithi031 ceph-mon[29457]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:47.332 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:47 smithi066 ceph-mon[33137]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:47.332 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:47 smithi066 systemd[1]: Starting Ceph mgr.y for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:11:47.580 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:47 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:47.291+0000 7fcff3f85200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:11:47.581 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:47 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:47.580+0000 7fcff3f85200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:11:47.582 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:47 smithi066 podman[34508]: 2024-04-25 13:11:47.332509419 +0000 UTC m=+0.114766076 container create 13a7b580e9ef0f03f5368820fe702b145d9e36db3a9e78901ca9faca950fc39b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T13:11:47.582 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:47 smithi066 podman[34508]: 2024-04-25 13:11:47.236338903 +0000 UTC m=+0.018595561 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:11:47.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.638+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/609482189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18103550 msgr2=0x7f8d18105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:47.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.638+0000 7f8d1dcb6700 1 --2- 172.21.15.31:0/609482189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18103550 0x7f8d18105940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009990 tx=0x7f8d08009ca0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.638+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/609482189 shutdown_connections 2024-04-25T13:11:47.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.638+0000 7f8d1dcb6700 1 --2- 172.21.15.31:0/609482189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d181087b0 0x7f8d1810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.638+0000 7f8d1dcb6700 1 --2- 172.21.15.31:0/609482189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18103550 0x7f8d18105940 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.638+0000 7f8d1dcb6700 1 --2- 172.21.15.31:0/609482189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d18105e80 0x7f8d18108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.638+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/609482189 >> 172.21.15.31:0/609482189 conn(0x7f8d180fcf60 msgr2=0x7f8d180ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:11:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.638+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/609482189 shutdown_connections 2024-04-25T13:11:47.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.638+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/609482189 wait complete. 2024-04-25T13:11:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.639+0000 7f8d1dcb6700 1 Processor -- start 2024-04-25T13:11:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.639+0000 7f8d1dcb6700 1 -- start start 2024-04-25T13:11:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.639+0000 7f8d1dcb6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18103550 0x7f8d18102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.639+0000 7f8d1dcb6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d18105e80 0x7f8d18101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.639+0000 7f8d1dcb6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d181087b0 0x7f8d18101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.639+0000 7f8d1dcb6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1810e570 con 0x7f8d18103550 2024-04-25T13:11:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.639+0000 7f8d1dcb6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1810e3f0 con 0x7f8d181087b0 2024-04-25T13:11:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.639+0000 7f8d1dcb6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1810e6f0 con 0x7f8d18105e80 2024-04-25T13:11:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18103550 0x7f8d18102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18103550 0x7f8d18102eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59712/0 (socket says 172.21.15.31:59712) 2024-04-25T13:11:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d177fe700 1 -- 172.21.15.31:0/28889080 learned_addr learned my addr 172.21.15.31:0/28889080 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:11:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d16ffd700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d18105e80 0x7f8d18101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d17fff700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d181087b0 0x7f8d18101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d177fe700 1 -- 172.21.15.31:0/28889080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d18105e80 msgr2=0x7f8d18101500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d177fe700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d18105e80 0x7f8d18101500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d177fe700 1 -- 172.21.15.31:0/28889080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d181087b0 msgr2=0x7f8d18101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d177fe700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d181087b0 0x7f8d18101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d177fe700 1 -- 172.21.15.31:0/28889080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d181022f0 con 0x7f8d18103550 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d177fe700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18103550 0x7f8d18102eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800b650 tx=0x7f8d0800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d14ff9700 1 -- 172.21.15.31:0/28889080 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08027070 con 0x7f8d18103550 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/28889080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d181a2ce0 con 0x7f8d18103550 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d14ff9700 1 -- 172.21.15.31:0/28889080 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0800be40 con 0x7f8d18103550 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d14ff9700 1 -- 172.21.15.31:0/28889080 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08017650 con 0x7f8d18103550 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.640+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/28889080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d181a3130 con 0x7f8d18103550 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.641+0000 7f8d14ff9700 1 -- 172.21.15.31:0/28889080 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8d0802a070 con 0x7f8d18103550 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.641+0000 7f8d14ff9700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8d00039890 0x7f8d0003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.641+0000 7f8d14ff9700 1 -- 172.21.15.31:0/28889080 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8d0800fdb0 con 0x7f8d18103550 2024-04-25T13:11:47.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.642+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/28889080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d18069730 con 0x7f8d18103550 2024-04-25T13:11:47.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.642+0000 7f8d16ffd700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8d00039890 0x7f8d0003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:11:47.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.642+0000 7f8d16ffd700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8d00039890 0x7f8d0003bd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005950 tx=0x7f8d0c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:11:47.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.647+0000 7f8d14ff9700 1 -- 172.21.15.31:0/28889080 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d08031070 con 0x7f8d18103550 2024-04-25T13:11:47.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:47.873+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/28889080 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8d18062f40 con 0x7f8d00039890 2024-04-25T13:11:47.877 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:47 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:47.672+0000 7fcff3f85200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:11:47.877 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:47 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:47.877+0000 7fcff3f85200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:11:48.015 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:47 smithi066 podman[34508]: 2024-04-25 13:11:47.659936789 +0000 UTC m=+0.442193447 container init 13a7b580e9ef0f03f5368820fe702b145d9e36db3a9e78901ca9faca950fc39b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T13:11:48.015 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:47 smithi066 podman[34508]: 2024-04-25 13:11:47.662788728 +0000 UTC m=+0.445045388 container start 13a7b580e9ef0f03f5368820fe702b145d9e36db3a9e78901ca9faca950fc39b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T13:11:48.015 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:47 smithi066 ceph-mgr[34527]: -- 172.21.15.66:0/504091840 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e992fe5a0 con 0x558e992d2800 2024-04-25T13:11:48.015 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:47 smithi066 bash[34508]: 13a7b580e9ef0f03f5368820fe702b145d9e36db3a9e78901ca9faca950fc39b 2024-04-25T13:11:48.015 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:47 smithi066 systemd[1]: Started Ceph mgr.y for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:11:48.015 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:47 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:47.822+0000 7fc946a29200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:11:48.015 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:48.013+0000 7fc946a29200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:11:48.218 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:48.072+0000 7fcff3f85200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:11:48.407 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:48.123+0000 7fc946a29200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:11:48.814 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:48.454+0000 7fc946a29200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:11:48.815 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:48.549+0000 7fc946a29200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:11:48.968 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:48.705+0000 7fcff3f85200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:11:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-mon[33038]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-mon[33038]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:11:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:11:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:48.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:49.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-mon[33137]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:49.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-mon[33137]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:11:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:11:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:49.209 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:48 smithi031 ceph-mon[29457]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:49.209 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:49.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:49.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:49.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:49.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:48 smithi031 ceph-mon[29457]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:11:49.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:11:49.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:11:49.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:49.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:49.449 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:49 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:49.139+0000 7fcff3f85200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:11:49.449 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:49 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:49.245+0000 7fcff3f85200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:11:49.449 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:49 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:49.449+0000 7fcff3f85200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:11:50.157 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:49 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:49.852+0000 7fc946a29200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:11:50.157 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:50.027+0000 7fc946a29200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:11:50.264 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:49 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:49.982+0000 7fcff3f85200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:11:50.264 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:50.074+0000 7fcff3f85200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:11:50.505 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:50.203+0000 7fc946a29200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:11:50.505 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:50.504+0000 7fc946a29200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:11:50.505 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.505 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.505 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.505 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.505 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:50.506 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:50.506 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.506 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:11:50.506 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:11:50.506 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:50.548 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:50.319+0000 7fcff3f85200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:11:50.548 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:50.452+0000 7fcff3f85200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:11:50.548 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:50.547+0000 7fcff3f85200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:11:50.548 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.548 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.548 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.548 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.548 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:50.548 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:50.549 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.549 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:11:50.549 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:11:50.549 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:50.665 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:50 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.665 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:50 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.665 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:50 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.665 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:50 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.665 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:50 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:50.665 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:50 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:50.666 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:50 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:50.666 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:50 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:11:50.666 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:50 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:11:50.666 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:50 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:50.812 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:50.599+0000 7fc946a29200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:11:50.812 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:50.811+0000 7fc946a29200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:11:50.968 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:50.663+0000 7fcff3f85200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:11:51.157 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:51 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:51.009+0000 7fc946a29200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:11:51.299 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:50.992+0000 7fcff3f85200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:11:51.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:51 smithi136 ceph-mon[33038]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:11:51.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:51 smithi136 ceph-mon[33038]: Reconfiguring daemon mgr.x on smithi031 2024-04-25T13:11:51.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:51 smithi136 ceph-mon[33038]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:51.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:51 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2228362980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5630157c-602a-4837-a0e0-026f57ea83b0"}]: dispatch 2024-04-25T13:11:51.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:51 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2228362980' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5630157c-602a-4837-a0e0-026f57ea83b0"}]': finished 2024-04-25T13:11:51.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:51 smithi136 ceph-mon[33038]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:11:51.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:51 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:11:51.593 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:51 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1324172827' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:11:51.637 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:51 smithi066 ceph-mon[33137]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:11:51.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:51 smithi066 ceph-mon[33137]: Reconfiguring daemon mgr.x on smithi031 2024-04-25T13:11:51.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:51 smithi066 ceph-mon[33137]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:51.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:51 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2228362980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5630157c-602a-4837-a0e0-026f57ea83b0"}]: dispatch 2024-04-25T13:11:51.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:51 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2228362980' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5630157c-602a-4837-a0e0-026f57ea83b0"}]': finished 2024-04-25T13:11:51.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:51 smithi066 ceph-mon[33137]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:11:51.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:51 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:11:51.638 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:51 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1324172827' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:11:51.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:51 smithi031 ceph-mon[29457]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:11:51.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:51 smithi031 ceph-mon[29457]: Reconfiguring daemon mgr.x on smithi031 2024-04-25T13:11:51.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:51 smithi031 ceph-mon[29457]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:51.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:51 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2228362980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5630157c-602a-4837-a0e0-026f57ea83b0"}]: dispatch 2024-04-25T13:11:51.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:51 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2228362980' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5630157c-602a-4837-a0e0-026f57ea83b0"}]': finished 2024-04-25T13:11:51.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:51 smithi031 ceph-mon[29457]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:11:51.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:51 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:11:51.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:51 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1324172827' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:11:51.907 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:51 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:51.636+0000 7fc946a29200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:11:51.969 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:51.592+0000 7fcff3f85200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:11:51.969 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:11:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:11:51.806+0000 7fcff3f85200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:11:52.383 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:52 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:52.075+0000 7fc946a29200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:11:52.383 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:52 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:52.173+0000 7fc946a29200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:11:52.383 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:52 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:52.382+0000 7fc946a29200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:11:52.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:52.774+0000 7f8d14ff9700 1 -- 172.21.15.31:0/28889080 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f8d08009d10 con 0x7f8d18103550 2024-04-25T13:11:53.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:52 smithi066 ceph-mon[33137]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:53.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:53.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:52 smithi066 ceph-mon[33137]: Standby manager daemon z started 2024-04-25T13:11:53.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:52 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:11:53.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:52 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:11:53.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:52 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:11:53.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:52 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:11:53.022 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:52 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:52.926+0000 7fc946a29200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:11:53.022 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:53.020+0000 7fc946a29200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:11:53.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:52 smithi136 ceph-mon[33038]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:53.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:53.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:52 smithi136 ceph-mon[33038]: Standby manager daemon z started 2024-04-25T13:11:53.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:52 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:11:53.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:52 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:11:53.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:52 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:11:53.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:52 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:11:53.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:52 smithi031 ceph-mon[29457]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:53.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:52 smithi031 ceph-mon[29457]: Standby manager daemon z started 2024-04-25T13:11:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:52 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:11:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:52 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:11:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:52 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:11:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:52 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.136:0/1735970593' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:11:53.628 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:53.272+0000 7fc946a29200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:11:53.628 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:53.411+0000 7fc946a29200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:11:53.628 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:53.509+0000 7fc946a29200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:11:53.628 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:53.627+0000 7fc946a29200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:11:53.895 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:53 smithi066 ceph-mon[33137]: mgrmap e14: x(active, since 81s), standbys: z 2024-04-25T13:11:53.895 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:11:54.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:53 smithi031 ceph-mon[29457]: mgrmap e14: x(active, since 81s), standbys: z 2024-04-25T13:11:54.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:11:54.157 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:53.960+0000 7fc946a29200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:11:54.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:53 smithi136 ceph-mon[33038]: mgrmap e14: x(active, since 81s), standbys: z 2024-04-25T13:11:54.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:11:54.778 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:54 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:54.594+0000 7fc946a29200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:11:55.041 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:54 smithi031 ceph-mon[29457]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:55.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:54 smithi066 ceph-mon[33137]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:55.158 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:11:54 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:11:54.817+0000 7fc946a29200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:11:55.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:54 smithi136 ceph-mon[33038]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:11:55.789+0000 7f8d14ff9700 1 -- 172.21.15.31:0/28889080 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f8d08020370 con 0x7f8d18103550 2024-04-25T13:11:55.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:55 smithi031 ceph-mon[29457]: Standby manager daemon y started 2024-04-25T13:11:55.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:55 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:11:55.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:55 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:11:55.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:55 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:11:55.950 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:55 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:11:56.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:55 smithi066 ceph-mon[33137]: Standby manager daemon y started 2024-04-25T13:11:56.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:55 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:11:56.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:55 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:11:56.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:55 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:11:56.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:55 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:11:56.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:55 smithi136 ceph-mon[33038]: Standby manager daemon y started 2024-04-25T13:11:56.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:55 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:11:56.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:55 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:11:56.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:55 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:11:56.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:55 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.66:0/437897877' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:11:57.090 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:56 smithi031 ceph-mon[29457]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:57.090 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:56 smithi031 ceph-mon[29457]: mgrmap e15: x(active, since 84s), standbys: z, y 2024-04-25T13:11:57.090 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:56 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:11:57.090 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:56 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:57.090 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:56 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:57.091 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:56 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:57.091 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:56 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:57.091 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:56 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:57.091 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:56 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:57.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:56 smithi066 ceph-mon[33137]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:56 smithi066 ceph-mon[33137]: mgrmap e15: x(active, since 84s), standbys: z, y 2024-04-25T13:11:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:56 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:11:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:56 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:56 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:56 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:56 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:56 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:56 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:57.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:56 smithi136 ceph-mon[33038]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:56 smithi136 ceph-mon[33038]: mgrmap e15: x(active, since 84s), standbys: z, y 2024-04-25T13:11:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:56 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:11:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:56 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:56 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:56 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:11:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:56 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:56 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:11:57.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:56 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:11:59.094 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:58 smithi031 ceph-mon[29457]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:59.094 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:58 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:11:59.094 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:58 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:59.094 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:11:58 smithi031 ceph-mon[29457]: Deploying daemon osd.0 on smithi031 2024-04-25T13:11:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:58 smithi066 ceph-mon[33137]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:58 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:11:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:58 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:11:58 smithi066 ceph-mon[33137]: Deploying daemon osd.0 on smithi031 2024-04-25T13:11:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:58 smithi136 ceph-mon[33038]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:11:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:58 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:11:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:58 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:11:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:11:58 smithi136 ceph-mon[33038]: Deploying daemon osd.0 on smithi031 2024-04-25T13:12:00.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:00 smithi031 ceph-mon[29457]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:01.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:00 smithi066 ceph-mon[33137]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:01.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:00 smithi136 ceph-mon[33038]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:02 smithi031 ceph-mon[29457]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:02 smithi066 ceph-mon[33137]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:03.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:02 smithi136 ceph-mon[33038]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:05.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:04 smithi136 ceph-mon[33038]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:05.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:05.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:04 smithi031 ceph-mon[29457]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:05.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:05.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:05.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:05.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:04 smithi066 ceph-mon[33137]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:05.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:05.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:05.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:07.185 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:06 smithi031 ceph-mon[29457]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:07.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:06 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:07.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:06 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:07.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:06 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:07.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:06 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:07.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:06 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:07.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:06 smithi136 ceph-mon[33038]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:07.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:06 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:07.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:06 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:07.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:06 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:07.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:06 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:07.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:06 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:07.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:06 smithi066 ceph-mon[33137]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:07.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:06 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:07.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:06 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:07.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:06 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:07.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:06 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:07.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:06 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:07.849 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2024-04-25T13:12:07.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.848+0000 7f8d14ff9700 1 -- 172.21.15.31:0/28889080 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8d18062f40 con 0x7f8d00039890 2024-04-25T13:12:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/28889080 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8d00039890 msgr2=0x7f8d0003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8d00039890 0x7f8d0003bd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005950 tx=0x7f8d0c00b410 comp rx=0 tx=0).stop 2024-04-25T13:12:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/28889080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18103550 msgr2=0x7f8d18102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18103550 0x7f8d18102eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800b650 tx=0x7f8d0800f690 comp rx=0 tx=0).stop 2024-04-25T13:12:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/28889080 shutdown_connections 2024-04-25T13:12:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d181087b0 0x7f8d18101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d18103550 0x7f8d18102eb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8d00039890 0x7f8d0003bd50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 --2- 172.21.15.31:0/28889080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d18105e80 0x7f8d18101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/28889080 >> 172.21.15.31:0/28889080 conn(0x7f8d180fcf60 msgr2=0x7f8d18105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:07.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/28889080 shutdown_connections 2024-04-25T13:12:07.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:12:07.850+0000 7f8d1dcb6700 1 -- 172.21.15.31:0/28889080 wait complete. 2024-04-25T13:12:08.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:07 smithi031 ceph-mon[29457]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:12:08.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:07 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:08.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:07 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:08.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:07 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:08.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:07 smithi136 ceph-mon[33038]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:12:08.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:07 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:08.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:07 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:08.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:07 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:08.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:07 smithi066 ceph-mon[33137]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:12:08.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:07 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:08.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:07 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:08.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:07 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:08.458 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T13:12:08.460 INFO:tasks.cephadm:Deploying osd.1 on smithi066 with /dev/vg_nvme/lv_4... 2024-04-25T13:12:08.461 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:12:08.723 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:12:08.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:08 smithi031 ceph-mon[29457]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:08.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:08 smithi031 ceph-mon[29457]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:12:08.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:08 smithi031 ceph-mon[29457]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:12:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:08 smithi031 ceph-mon[29457]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:12:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:08 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:08.984 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:12:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0[44419]: 2024-04-25T13:12:08.940+0000 7f8eda4d7700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T13:12:08.984 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:12:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0[44419]: 2024-04-25T13:12:08.943+0000 7f8ed62ca700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:12:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:08 smithi066 ceph-mon[33137]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:08 smithi066 ceph-mon[33137]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:12:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:08 smithi066 ceph-mon[33137]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:12:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:08 smithi066 ceph-mon[33137]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:12:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:08 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:09.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:08 smithi136 ceph-mon[33038]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:09.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:08 smithi136 ceph-mon[33038]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:12:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:08 smithi136 ceph-mon[33038]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:12:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:08 smithi136 ceph-mon[33038]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:12:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:08 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:10.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:09 smithi066 ceph-mon[33137]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:12:10.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:09 smithi066 ceph-mon[33137]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:12:10.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:10.067 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:10.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:09 smithi136 ceph-mon[33038]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:12:10.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:09 smithi136 ceph-mon[33038]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:12:10.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:10.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:10.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:09 smithi031 ceph-mon[29457]: from='osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:12:10.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:09 smithi031 ceph-mon[29457]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:12:10.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:10.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:10.636 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:12:10.636 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:12:10.636 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-04-25T13:12:10.637 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-04-25T13:12:10.637 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112081 s, 936 MB/s 2024-04-25T13:12:10.637 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 2024-04-25T13:12:10.637 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-04-25T13:12:11.197 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:10 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:12:11.197 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:10 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:12:11.197 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:10 smithi031 ceph-mon[29457]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:11.197 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:10 smithi031 ceph-mon[29457]: osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670] boot 2024-04-25T13:12:11.197 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:10 smithi031 ceph-mon[29457]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:12:11.197 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:11.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:10 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:12:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:10 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:12:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:10 smithi136 ceph-mon[33038]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:10 smithi136 ceph-mon[33038]: osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670] boot 2024-04-25T13:12:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:10 smithi136 ceph-mon[33038]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:12:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:10 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:11.367 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:10 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:12:11.367 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:10 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:12:11.367 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:10 smithi066 ceph-mon[33137]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:12:11.367 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:10 smithi066 ceph-mon[33137]: osd.0 [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670] boot 2024-04-25T13:12:11.367 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:10 smithi066 ceph-mon[33137]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:12:11.367 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:10 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:12:11.443 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_4 2024-04-25T13:12:11.690 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:12:12.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:11 smithi136 ceph-mon[33038]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:12:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:11 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi031 2024-04-25T13:12:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:11 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:11 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:11 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:12:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:11 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi031 to 17093M 2024-04-25T13:12:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:11 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:11 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:11 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:11 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:12.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:11 smithi031 ceph-mon[29457]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:12:12.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:11 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi031 2024-04-25T13:12:12.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:12.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:12.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:12:12.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:11 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi031 to 17093M 2024-04-25T13:12:12.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:12.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:12.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:12.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:12.393 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:11 smithi066 ceph-mon[33137]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:12:12.394 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:11 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi031 2024-04-25T13:12:12.394 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:11 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:12.394 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:11 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:12.394 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:11 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:12:12.394 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:11 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi031 to 17093M 2024-04-25T13:12:12.394 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:11 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:12.394 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:11 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:12.394 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:11 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:12.394 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:11 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:13.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.112+0000 7effd1a73700 1 -- 172.21.15.66:0/1272144134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effcc101530 msgr2=0x7effcc10b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.112+0000 7effd1a73700 1 --2- 172.21.15.66:0/1272144134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effcc101530 0x7effcc10b100 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effc400b0a0 tx=0x7effc400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.112+0000 7effd1a73700 1 -- 172.21.15.66:0/1272144134 shutdown_connections 2024-04-25T13:12:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.112+0000 7effd1a73700 1 --2- 172.21.15.66:0/1272144134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effcc101530 0x7effcc10b100 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.112+0000 7effd1a73700 1 --2- 172.21.15.66:0/1272144134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc104110 0x7effcc104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.112+0000 7effd1a73700 1 --2- 172.21.15.66:0/1272144134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effcc100b90 0x7effcc100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.112+0000 7effd1a73700 1 -- 172.21.15.66:0/1272144134 >> 172.21.15.66:0/1272144134 conn(0x7effcc0fc710 msgr2=0x7effcc0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.112+0000 7effd1a73700 1 -- 172.21.15.66:0/1272144134 shutdown_connections 2024-04-25T13:12:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.112+0000 7effd1a73700 1 -- 172.21.15.66:0/1272144134 wait complete. 2024-04-25T13:12:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.113+0000 7effd1a73700 1 Processor -- start 2024-04-25T13:12:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.113+0000 7effd1a73700 1 -- start start 2024-04-25T13:12:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.113+0000 7effd1a73700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effcc100b90 0x7effcc198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.113+0000 7effd1a73700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effcc101530 0x7effcc1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.113+0000 7effd1a73700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc104110 0x7effcc1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.113+0000 7effd1a73700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10e5f0 con 0x7effcc104110 2024-04-25T13:12:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.113+0000 7effd1a73700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10e470 con 0x7effcc101530 2024-04-25T13:12:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.113+0000 7effd1a73700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7effcc10e770 con 0x7effcc100b90 2024-04-25T13:12:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.113+0000 7effcaffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effcc101530 0x7effcc1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effcaffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effcc101530 0x7effcc1990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36470/0 (socket says 172.21.15.66:36470) 2024-04-25T13:12:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effcaffd700 1 -- 172.21.15.66:0/916379238 learned_addr learned my addr 172.21.15.66:0/916379238 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-25T13:12:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effcbfff700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc104110 0x7effcc1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effcb7fe700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effcc100b90 0x7effcc198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effcaffd700 1 -- 172.21.15.66:0/916379238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effcc100b90 msgr2=0x7effcc198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effcaffd700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effcc100b90 0x7effcc198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effcaffd700 1 -- 172.21.15.66:0/916379238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc104110 msgr2=0x7effcc1a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effcaffd700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc104110 0x7effcc1a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effcaffd700 1 -- 172.21.15.66:0/916379238 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc1a0810 con 0x7effcc101530 2024-04-25T13:12:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effcaffd700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effcc101530 0x7effcc1990f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effc000d980 tx=0x7effc000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effc8ff9700 1 -- 172.21.15.66:0/916379238 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0016070 con 0x7effcc101530 2024-04-25T13:12:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effc8ff9700 1 -- 172.21.15.66:0/916379238 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc0004830 con 0x7effcc101530 2024-04-25T13:12:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effc8ff9700 1 -- 172.21.15.66:0/916379238 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0005520 con 0x7effcc101530 2024-04-25T13:12:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effd1a73700 1 -- 172.21.15.66:0/916379238 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effcc1a0b00 con 0x7effcc101530 2024-04-25T13:12:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.114+0000 7effd1a73700 1 -- 172.21.15.66:0/916379238 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effcc1a0ff0 con 0x7effcc101530 2024-04-25T13:12:13.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.115+0000 7effd1a73700 1 -- 172.21.15.66:0/916379238 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effcc069730 con 0x7effcc101530 2024-04-25T13:12:13.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.116+0000 7effc8ff9700 1 -- 172.21.15.66:0/916379238 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7effc0004220 con 0x7effcc101530 2024-04-25T13:12:13.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.117+0000 7effc8ff9700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7effb40a3c30 0x7effb40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:13.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.117+0000 7effc8ff9700 1 -- 172.21.15.66:0/916379238 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7effc00ce790 con 0x7effcc101530 2024-04-25T13:12:13.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.117+0000 7effcb7fe700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7effb40a3c30 0x7effb40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:13.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.118+0000 7effcb7fe700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7effb40a3c30 0x7effb40a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.122+0000 7effc8ff9700 1 -- 172.21.15.66:0/916379238 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effc001f020 con 0x7effcc101530 2024-04-25T13:12:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:13.339+0000 7effd1a73700 1 -- 172.21.15.66:0/916379238 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7effcc062e80 con 0x7effb40a3c30 2024-04-25T13:12:13.340 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:13 smithi066 ceph-mon[33137]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:13.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:13 smithi136 ceph-mon[33038]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:13.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:13 smithi031 ceph-mon[29457]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:14.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:14 smithi066 ceph-mon[33137]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:14.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:12:14.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:12:14.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:14.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:14 smithi136 ceph-mon[33038]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:14.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:12:14.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:12:14.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:14.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:14 smithi031 ceph-mon[29457]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:14.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:12:14.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:12:14.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:15.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:15 smithi066 ceph-mon[33137]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:15.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:15 smithi136 ceph-mon[33038]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:15.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:15 smithi031 ceph-mon[29457]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:16.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:16 smithi066 ceph-mon[33137]: from='client.? 172.21.15.66:0/4033879148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b3a9a6-6981-4000-a44d-f4ce49325ed6"}]: dispatch 2024-04-25T13:12:16.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:16 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b3a9a6-6981-4000-a44d-f4ce49325ed6"}]: dispatch 2024-04-25T13:12:16.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:16 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5b3a9a6-6981-4000-a44d-f4ce49325ed6"}]': finished 2024-04-25T13:12:16.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:16 smithi066 ceph-mon[33137]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:12:16.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:16 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:16.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:16 smithi136 ceph-mon[33038]: from='client.? 172.21.15.66:0/4033879148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b3a9a6-6981-4000-a44d-f4ce49325ed6"}]: dispatch 2024-04-25T13:12:16.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:16 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b3a9a6-6981-4000-a44d-f4ce49325ed6"}]: dispatch 2024-04-25T13:12:16.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:16 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5b3a9a6-6981-4000-a44d-f4ce49325ed6"}]': finished 2024-04-25T13:12:16.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:16 smithi136 ceph-mon[33038]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:12:16.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:16 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:16.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:16 smithi031 ceph-mon[29457]: from='client.? 172.21.15.66:0/4033879148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b3a9a6-6981-4000-a44d-f4ce49325ed6"}]: dispatch 2024-04-25T13:12:16.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:16 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5b3a9a6-6981-4000-a44d-f4ce49325ed6"}]: dispatch 2024-04-25T13:12:16.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:16 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5b3a9a6-6981-4000-a44d-f4ce49325ed6"}]': finished 2024-04-25T13:12:16.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:16 smithi031 ceph-mon[29457]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:12:16.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:16 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:17.656 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:17 smithi031 ceph-mon[29457]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:17.656 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:17 smithi031 ceph-mon[29457]: from='client.? 172.21.15.66:0/3015759594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:12:17.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:17 smithi066 ceph-mon[33137]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:17.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:17 smithi066 ceph-mon[33137]: from='client.? 172.21.15.66:0/3015759594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:12:17.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:17 smithi136 ceph-mon[33038]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:17.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:17 smithi136 ceph-mon[33038]: from='client.? 172.21.15.66:0/3015759594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:12:19.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:19 smithi066 ceph-mon[33137]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:19.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:19 smithi136 ceph-mon[33038]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:19.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:19 smithi031 ceph-mon[29457]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:21 smithi066 ceph-mon[33137]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:21.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:21 smithi136 ceph-mon[33038]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:21.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:21 smithi031 ceph-mon[29457]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:23.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:23 smithi066 ceph-mon[33137]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:23.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:23 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:12:23.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:23 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:23.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:23 smithi136 ceph-mon[33038]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:23.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:23 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:12:23.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:23 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:23.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:23 smithi031 ceph-mon[29457]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:23.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:23 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:12:23.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:23 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:24.536 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:24 smithi066 ceph-mon[33137]: Deploying daemon osd.1 on smithi066 2024-04-25T13:12:24.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:24 smithi031 ceph-mon[29457]: Deploying daemon osd.1 on smithi066 2024-04-25T13:12:24.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:24 smithi136 ceph-mon[33038]: Deploying daemon osd.1 on smithi066 2024-04-25T13:12:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:25 smithi066 ceph-mon[33137]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:25.652 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:25 smithi031 ceph-mon[29457]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:25.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:25 smithi136 ceph-mon[33038]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:27.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:27 smithi066 ceph-mon[33137]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:27.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:27 smithi136 ceph-mon[33038]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:27.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:27 smithi031 ceph-mon[29457]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:29.373 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:29 smithi066 ceph-mon[33137]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:29.373 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:29 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:29.373 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:29 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:29.373 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:29 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:29.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:29 smithi136 ceph-mon[33038]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:29.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:29 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:29.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:29 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:29.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:29 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:29.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:29 smithi031 ceph-mon[29457]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:29.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:29 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:29.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:29 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:29.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:29 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:31.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:31 smithi066 ceph-mon[33137]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:31.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:31.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:31.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:31.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:31.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:31.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:31 smithi136 ceph-mon[33038]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:31.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:31.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:31.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:31.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:31.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:31.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:31 smithi031 ceph-mon[29457]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:31.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:31.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:31.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:31.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:31.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:32.508 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 1 on host 'smithi066' 2024-04-25T13:12:32.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.508+0000 7effc8ff9700 1 -- 172.21.15.66:0/916379238 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7effcc062e80 con 0x7effb40a3c30 2024-04-25T13:12:32.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.510+0000 7effd1a73700 1 -- 172.21.15.66:0/916379238 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7effb40a3c30 msgr2=0x7effb40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:32.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.510+0000 7effd1a73700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7effb40a3c30 0x7effb40a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc008040 comp rx=0 tx=0).stop 2024-04-25T13:12:32.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.511+0000 7effd1a73700 1 -- 172.21.15.66:0/916379238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effcc101530 msgr2=0x7effcc1990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:32.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.511+0000 7effd1a73700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effcc101530 0x7effcc1990f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effc000d980 tx=0x7effc000dc90 comp rx=0 tx=0).stop 2024-04-25T13:12:32.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.511+0000 7effd1a73700 1 -- 172.21.15.66:0/916379238 shutdown_connections 2024-04-25T13:12:32.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.511+0000 7effd1a73700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7effb40a3c30 0x7effb40a60f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:32.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.511+0000 7effd1a73700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effcc100b90 0x7effcc198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:32.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.511+0000 7effd1a73700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effcc104110 0x7effcc1a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:32.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.511+0000 7effd1a73700 1 --2- 172.21.15.66:0/916379238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effcc101530 0x7effcc1990f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:32.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.511+0000 7effd1a73700 1 -- 172.21.15.66:0/916379238 >> 172.21.15.66:0/916379238 conn(0x7effcc0fc710 msgr2=0x7effcc107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:32.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.511+0000 7effd1a73700 1 -- 172.21.15.66:0/916379238 shutdown_connections 2024-04-25T13:12:32.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:12:32.511+0000 7effd1a73700 1 -- 172.21.15.66:0/916379238 wait complete. 2024-04-25T13:12:33.153 DEBUG:teuthology.orchestra.run.smithi066:osd.1> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T13:12:33.156 INFO:tasks.cephadm:Deploying osd.2 on smithi136 with /dev/vg_nvme/lv_4... 2024-04-25T13:12:33.156 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:12:33.397 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:12:33.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:33 smithi066 ceph-mon[33137]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:33.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:33 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:33 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:33 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:33 smithi066 ceph-mon[33137]: from='osd.1 [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:12:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:33 smithi066 ceph-mon[33137]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:12:33.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:33 smithi136 ceph-mon[33038]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:33.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:33.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:33 smithi136 ceph-mon[33038]: from='osd.1 [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:12:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:33 smithi136 ceph-mon[33038]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:12:33.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:33 smithi031 ceph-mon[29457]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:33.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:33.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:33.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:33.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:33 smithi031 ceph-mon[29457]: from='osd.1 [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:12:33.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:33 smithi031 ceph-mon[29457]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:12:34.517 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:34 smithi066 ceph-mon[33137]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:12:34.517 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:34 smithi066 ceph-mon[33137]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:12:34.518 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:34 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:34.518 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:34 smithi066 ceph-mon[33137]: from='osd.1 [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:12:34.518 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:34 smithi066 ceph-mon[33137]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:12:34.518 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:34 smithi066 ceph-mon[33137]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:34.518 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:12:34 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1[37750]: 2024-04-25T13:12:34.493+0000 7efd891dd700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T13:12:34.518 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:12:34 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1[37750]: 2024-04-25T13:12:34.497+0000 7efd84fd0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:12:34.688 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:34 smithi136 ceph-mon[33038]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:12:34.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:34 smithi136 ceph-mon[33038]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:12:34.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:34 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:34.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:34 smithi136 ceph-mon[33038]: from='osd.1 [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:12:34.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:34 smithi136 ceph-mon[33038]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:12:34.689 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:34 smithi136 ceph-mon[33038]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:34.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:34 smithi031 ceph-mon[29457]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:12:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:34 smithi031 ceph-mon[29457]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:12:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:34 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:34 smithi031 ceph-mon[29457]: from='osd.1 [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:12:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:34 smithi031 ceph-mon[29457]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:12:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:34 smithi031 ceph-mon[29457]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:35.271 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:12:35.271 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:12:35.272 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-04-25T13:12:35.272 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-04-25T13:12:35.272 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00843914 s, 1.2 GB/s 2024-04-25T13:12:35.272 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-04-25T13:12:35.599 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:35 smithi136 ceph-mon[33038]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:12:35.599 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:35 smithi136 ceph-mon[33038]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:12:35.599 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:35.599 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:35.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:35 smithi066 ceph-mon[33137]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:12:35.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:35 smithi066 ceph-mon[33137]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:12:35.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:35.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:35.878 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_4 2024-04-25T13:12:35.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:35 smithi031 ceph-mon[29457]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:12:35.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:35 smithi031 ceph-mon[29457]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:12:35.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:35.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:36.115 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:12:36.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: osd.1 [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860] boot 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi066 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi066 to 17088M 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:36.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:36.912 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:12:36.912 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:12:36.912 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: osd.1 [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860] boot 2024-04-25T13:12:36.912 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:12:36.912 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:36.912 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:36.912 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi066 2024-04-25T13:12:36.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:36.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:36.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:12:36.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi066 to 17088M 2024-04-25T13:12:36.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:36.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:36.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:36.913 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:36.913 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: osd.1 [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860] boot 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi066 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi066 to 17088M 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:36.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:36.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:36.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:37.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.662+0000 7f262111a700 1 -- 172.21.15.136:0/2685877412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f261c103ca0 msgr2=0x7f261c104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:37.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.662+0000 7f262111a700 1 --2- 172.21.15.136:0/2685877412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f261c103ca0 0x7f261c104100 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f260c009990 tx=0x7f260c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:12:37.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.662+0000 7f262111a700 1 -- 172.21.15.136:0/2685877412 shutdown_connections 2024-04-25T13:12:37.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.662+0000 7f262111a700 1 --2- 172.21.15.136:0/2685877412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c104640 0x7f261c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:37.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.662+0000 7f262111a700 1 --2- 172.21.15.136:0/2685877412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261c102aa0 0x7f261c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:37.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.662+0000 7f262111a700 1 --2- 172.21.15.136:0/2685877412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f261c103ca0 0x7f261c104100 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:37.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.662+0000 7f262111a700 1 -- 172.21.15.136:0/2685877412 >> 172.21.15.136:0/2685877412 conn(0x7f261c0fe040 msgr2=0x7f261c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:37.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.662+0000 7f262111a700 1 -- 172.21.15.136:0/2685877412 shutdown_connections 2024-04-25T13:12:37.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.662+0000 7f262111a700 1 -- 172.21.15.136:0/2685877412 wait complete. 2024-04-25T13:12:37.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.663+0000 7f262111a700 1 Processor -- start 2024-04-25T13:12:37.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.663+0000 7f262111a700 1 -- start start 2024-04-25T13:12:37.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.663+0000 7f262111a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f261c102aa0 0x7f261c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:37.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.663+0000 7f262111a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c103ca0 0x7f261c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.663+0000 7f262111a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261c104640 0x7f261c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.663+0000 7f262111a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10e540 con 0x7f261c104640 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.663+0000 7f262111a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10e3c0 con 0x7f261c103ca0 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.663+0000 7f262111a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10e6c0 con 0x7f261c102aa0 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.663+0000 7f261ad9d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f261c102aa0 0x7f261c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.663+0000 7f261ad9d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f261c102aa0 0x7f261c198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:56536/0 (socket says 172.21.15.136:56536) 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.663+0000 7f261ad9d700 1 -- 172.21.15.136:0/192448864 learned_addr learned my addr 172.21.15.136:0/192448864 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f261a59c700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c103ca0 0x7f261c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f261b59e700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261c104640 0x7f261c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f261ad9d700 1 -- 172.21.15.136:0/192448864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c103ca0 msgr2=0x7f261c199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f261ad9d700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c103ca0 0x7f261c199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f261ad9d700 1 -- 172.21.15.136:0/192448864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261c104640 msgr2=0x7f261c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:37.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f261ad9d700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261c104640 0x7f261c1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:37.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f261ad9d700 1 -- 172.21.15.136:0/192448864 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c1a0840 con 0x7f261c102aa0 2024-04-25T13:12:37.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f261ad9d700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f261c102aa0 0x7f261c198be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f260400d9b0 tx=0x7f260400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:37.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f2613fff700 1 -- 172.21.15.136:0/192448864 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604017030 con 0x7f261c102aa0 2024-04-25T13:12:37.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f2613fff700 1 -- 172.21.15.136:0/192448864 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26040044d0 con 0x7f261c102aa0 2024-04-25T13:12:37.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f262111a700 1 -- 172.21.15.136:0/192448864 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261c1a0b30 con 0x7f261c102aa0 2024-04-25T13:12:37.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f2613fff700 1 -- 172.21.15.136:0/192448864 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26040038e0 con 0x7f261c102aa0 2024-04-25T13:12:37.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.664+0000 7f262111a700 1 -- 172.21.15.136:0/192448864 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261c1a1070 con 0x7f261c102aa0 2024-04-25T13:12:37.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.665+0000 7f262111a700 1 -- 172.21.15.136:0/192448864 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261c069730 con 0x7f261c102aa0 2024-04-25T13:12:37.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.666+0000 7f2613fff700 1 -- 172.21.15.136:0/192448864 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f260401a070 con 0x7f261c102aa0 2024-04-25T13:12:37.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.667+0000 7f2613fff700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f26080a3bb0 0x7f26080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:12:37.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.667+0000 7f2613fff700 1 -- 172.21.15.136:0/192448864 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f26040ce420 con 0x7f261c102aa0 2024-04-25T13:12:37.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.667+0000 7f261a59c700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f26080a3bb0 0x7f26080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:12:37.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.668+0000 7f261a59c700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f26080a3bb0 0x7f26080a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f260c00b5c0 tx=0x7f260c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:12:37.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.672+0000 7f2613fff700 1 -- 172.21.15.136:0/192448864 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f260409c650 con 0x7f261c102aa0 2024-04-25T13:12:37.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:37 smithi136 ceph-mon[33038]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:12:37.880 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:37.880+0000 7f262111a700 1 -- 172.21.15.136:0/192448864 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f261c062e80 con 0x7f26080a3bb0 2024-04-25T13:12:37.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:37 smithi066 ceph-mon[33137]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:12:37.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:37 smithi031 ceph-mon[29457]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:12:38.812 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:38 smithi136 ceph-mon[33038]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:38.812 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:38 smithi136 ceph-mon[33038]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:38.812 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:38 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:12:38.813 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:38 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:12:38.813 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:38 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:38.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:38 smithi066 ceph-mon[33137]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:38.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:38 smithi066 ceph-mon[33137]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:38.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:38 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:12:38.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:38 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:12:38.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:38 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:38.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:38 smithi031 ceph-mon[29457]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:38.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:38 smithi031 ceph-mon[29457]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:12:38.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:38 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:12:38.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:38 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:12:38.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:38 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:41.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:40 smithi066 ceph-mon[33137]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:40 smithi066 ceph-mon[33137]: from='client.? 172.21.15.136:0/2817080021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b167a2-c38c-4b5d-a02c-b155ef73624b"}]: dispatch 2024-04-25T13:12:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:40 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b167a2-c38c-4b5d-a02c-b155ef73624b"}]: dispatch 2024-04-25T13:12:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:40 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3b167a2-c38c-4b5d-a02c-b155ef73624b"}]': finished 2024-04-25T13:12:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:40 smithi066 ceph-mon[33137]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:12:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:12:41.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:40 smithi136 ceph-mon[33038]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:40 smithi136 ceph-mon[33038]: from='client.? 172.21.15.136:0/2817080021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b167a2-c38c-4b5d-a02c-b155ef73624b"}]: dispatch 2024-04-25T13:12:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:40 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b167a2-c38c-4b5d-a02c-b155ef73624b"}]: dispatch 2024-04-25T13:12:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:40 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3b167a2-c38c-4b5d-a02c-b155ef73624b"}]': finished 2024-04-25T13:12:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:40 smithi136 ceph-mon[33038]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:12:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:12:41.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:40 smithi031 ceph-mon[29457]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:41.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:40 smithi031 ceph-mon[29457]: from='client.? 172.21.15.136:0/2817080021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b167a2-c38c-4b5d-a02c-b155ef73624b"}]: dispatch 2024-04-25T13:12:41.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:40 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3b167a2-c38c-4b5d-a02c-b155ef73624b"}]: dispatch 2024-04-25T13:12:41.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:40 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3b167a2-c38c-4b5d-a02c-b155ef73624b"}]': finished 2024-04-25T13:12:41.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:40 smithi031 ceph-mon[29457]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:12:41.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:12:42.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:41 smithi066 ceph-mon[33137]: from='client.? 172.21.15.136:0/61443301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:12:42.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:41 smithi136 ceph-mon[33038]: from='client.? 172.21.15.136:0/61443301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:12:42.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:41 smithi031 ceph-mon[29457]: from='client.? 172.21.15.136:0/61443301' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:12:43.059 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:42 smithi066 ceph-mon[33137]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:43.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:42 smithi136 ceph-mon[33038]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:43.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:42 smithi031 ceph-mon[29457]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:45.023 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:44 smithi136 ceph-mon[33038]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:44 smithi066 ceph-mon[33137]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:45.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:44 smithi031 ceph-mon[29457]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:46.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:46 smithi136 ceph-mon[33038]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:47.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:46 smithi066 ceph-mon[33137]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:47.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:46 smithi031 ceph-mon[29457]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:48.914 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:48 smithi136 ceph-mon[33038]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:48.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:12:48.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:48.915 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:48 smithi136 ceph-mon[33038]: Deploying daemon osd.2 on smithi136 2024-04-25T13:12:49.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:48 smithi066 ceph-mon[33137]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:12:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:48 smithi066 ceph-mon[33137]: Deploying daemon osd.2 on smithi136 2024-04-25T13:12:49.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:48 smithi031 ceph-mon[29457]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:49.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:12:49.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:49.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:48 smithi031 ceph-mon[29457]: Deploying daemon osd.2 on smithi136 2024-04-25T13:12:50.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:50 smithi136 ceph-mon[33038]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:51.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:50 smithi066 ceph-mon[33137]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:51.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:50 smithi031 ceph-mon[29457]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:53.074 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:52 smithi136 ceph-mon[33038]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:53.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:52 smithi066 ceph-mon[33137]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:53.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:52 smithi031 ceph-mon[29457]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:54.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:54.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:54.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:54.683 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:54.683 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:54.683 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:54.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:54.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:54.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:55.586 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:55 smithi136 ceph-mon[33038]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:55.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:55 smithi066 ceph-mon[33137]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:55.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:55 smithi031 ceph-mon[29457]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:57.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.204+0000 7f2613fff700 1 -- 172.21.15.136:0/192448864 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f261c062e80 con 0x7f26080a3bb0 2024-04-25T13:12:57.205 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 2 on host 'smithi136' 2024-04-25T13:12:57.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.206+0000 7f262111a700 1 -- 172.21.15.136:0/192448864 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f26080a3bb0 msgr2=0x7f26080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:57.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.206+0000 7f262111a700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f26080a3bb0 0x7f26080a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f260c00b5c0 tx=0x7f260c005fb0 comp rx=0 tx=0).stop 2024-04-25T13:12:57.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.206+0000 7f262111a700 1 -- 172.21.15.136:0/192448864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f261c102aa0 msgr2=0x7f261c198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:12:57.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.206+0000 7f262111a700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f261c102aa0 0x7f261c198be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f260400d9b0 tx=0x7f260400dcc0 comp rx=0 tx=0).stop 2024-04-25T13:12:57.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.206+0000 7f262111a700 1 -- 172.21.15.136:0/192448864 shutdown_connections 2024-04-25T13:12:57.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.206+0000 7f262111a700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f26080a3bb0 0x7f26080a6070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:57.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.206+0000 7f262111a700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f261c102aa0 0x7f261c198be0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:57.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.206+0000 7f262111a700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f261c104640 0x7f261c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:57.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.206+0000 7f262111a700 1 --2- 172.21.15.136:0/192448864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f261c103ca0 0x7f261c199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:12:57.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.206+0000 7f262111a700 1 -- 172.21.15.136:0/192448864 >> 172.21.15.136:0/192448864 conn(0x7f261c0fe040 msgr2=0x7f261c1070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:12:57.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.207+0000 7f262111a700 1 -- 172.21.15.136:0/192448864 shutdown_connections 2024-04-25T13:12:57.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:12:57.207+0000 7f262111a700 1 -- 172.21.15.136:0/192448864 wait complete. 2024-04-25T13:12:57.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:57 smithi136 ceph-mon[33038]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:57.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:57.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:57.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:57.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:57 smithi136 ceph-mon[33038]: from='osd.2 [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:12:57.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:57 smithi136 ceph-mon[33038]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:12:57.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:57 smithi066 ceph-mon[33137]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:57.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:57.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:57.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:57.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:57 smithi066 ceph-mon[33137]: from='osd.2 [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:12:57.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:57 smithi066 ceph-mon[33137]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:12:57.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:57 smithi031 ceph-mon[29457]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:12:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:12:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:12:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:12:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:57 smithi031 ceph-mon[29457]: from='osd.2 [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:12:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:57 smithi031 ceph-mon[29457]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:12:57.833 DEBUG:teuthology.orchestra.run.smithi136:osd.2> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T13:12:57.835 INFO:tasks.cephadm:Deploying osd.3 on smithi031 with /dev/vg_nvme/lv_3... 2024-04-25T13:12:57.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:12:58.090 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:12:59.425 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:59 smithi136 ceph-mon[33038]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:59.425 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:59 smithi136 ceph-mon[33038]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:12:59.426 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:59 smithi136 ceph-mon[33038]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:12:59.426 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:59 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:12:59.426 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:59 smithi136 ceph-mon[33038]: from='osd.2 [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:12:59.426 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:12:59 smithi136 ceph-mon[33038]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:12:59.426 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:12:59 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2[38228]: 2024-04-25T13:12:59.168+0000 7f1e43cd4700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T13:12:59.426 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:12:59 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2[38228]: 2024-04-25T13:12:59.173+0000 7f1e3fac7700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:12:59.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:59 smithi031 ceph-mon[29457]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:59.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:59 smithi031 ceph-mon[29457]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:12:59.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:59 smithi031 ceph-mon[29457]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:12:59.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:59 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:12:59.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:59 smithi031 ceph-mon[29457]: from='osd.2 [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:12:59.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:12:59 smithi031 ceph-mon[29457]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:12:59.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:59 smithi066 ceph-mon[33137]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:12:59.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:59 smithi066 ceph-mon[33137]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:12:59.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:59 smithi066 ceph-mon[33137]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:12:59.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:59 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:12:59.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:59 smithi066 ceph-mon[33137]: from='osd.2 [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:12:59.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:12:59 smithi066 ceph-mon[33137]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:13:00.126 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:13:00.126 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:13:00.126 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T13:13:00.126 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T13:13:00.126 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126008 s, 832 MB/s 2024-04-25T13:13:00.126 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T13:13:00.313 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:00 smithi136 ceph-mon[33038]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:13:00.314 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:00 smithi136 ceph-mon[33038]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:13:00.314 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:00 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:00.314 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:00 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:00.432 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:00 smithi031 ceph-mon[29457]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:13:00.432 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:00 smithi031 ceph-mon[29457]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:13:00.432 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:00 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:00.433 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:00 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:00.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:00 smithi066 ceph-mon[33137]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:13:00.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:00 smithi066 ceph-mon[33137]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:13:00.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:00 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:00.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:00 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:00.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2024-04-25T13:13:01.027 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:13:01.422 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: osd.2 [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269] boot 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:01.423 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:01 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:13:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:13:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:13:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: osd.2 [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269] boot 2024-04-25T13:13:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:13:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:01.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:01.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:01.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:01 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:01.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:13:01.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:13:01.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:13:01.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:01.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: osd.2 [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269] boot 2024-04-25T13:13:01.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:13:01.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:13:01.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:01.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:01.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:01.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:01.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:01.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:01.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:01 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:02.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:02 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi136 2024-04-25T13:13:02.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:02 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi136 to 17088M 2024-04-25T13:13:02.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:02 smithi031 ceph-mon[29457]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:13:02.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:13:02.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:02 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi136 2024-04-25T13:13:02.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:02 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi136 to 17088M 2024-04-25T13:13:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:02 smithi136 ceph-mon[33038]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:13:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:13:02.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.609+0000 7fb25614b700 1 -- 172.21.15.31:0/1391002691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb250102ad0 msgr2=0x7fb250102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.609+0000 7fb25614b700 1 --2- 172.21.15.31:0/1391002691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb250102ad0 0x7fb250102ef0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb2400099c0 tx=0x7fb240009cd0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.609+0000 7fb25614b700 1 -- 172.21.15.31:0/1391002691 shutdown_connections 2024-04-25T13:13:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.609+0000 7fb25614b700 1 --2- 172.21.15.31:0/1391002691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb250104670 0x7fb25010af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.609+0000 7fb25614b700 1 --2- 172.21.15.31:0/1391002691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb250102ad0 0x7fb250102ef0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.609+0000 7fb25614b700 1 --2- 172.21.15.31:0/1391002691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250103cd0 0x7fb250104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.609+0000 7fb25614b700 1 -- 172.21.15.31:0/1391002691 >> 172.21.15.31:0/1391002691 conn(0x7fb2500fe090 msgr2=0x7fb2501004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.609+0000 7fb25614b700 1 -- 172.21.15.31:0/1391002691 shutdown_connections 2024-04-25T13:13:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.609+0000 7fb25614b700 1 -- 172.21.15.31:0/1391002691 wait complete. 2024-04-25T13:13:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.609+0000 7fb25614b700 1 Processor -- start 2024-04-25T13:13:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.609+0000 7fb25614b700 1 -- start start 2024-04-25T13:13:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb25614b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb250102ad0 0x7fb250198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb25614b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb250103cd0 0x7fb2501991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb25614b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250104670 0x7fb2501a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb25614b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb25010e5b0 con 0x7fb250102ad0 2024-04-25T13:13:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb25614b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb25010e430 con 0x7fb250104670 2024-04-25T13:13:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb25614b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb25010e730 con 0x7fb250103cd0 2024-04-25T13:13:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb24f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb250102ad0 0x7fb250198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb24f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb250102ad0 0x7fb250198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44342/0 (socket says 172.21.15.31:44342) 2024-04-25T13:13:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb24f7fe700 1 -- 172.21.15.31:0/1265492559 learned_addr learned my addr 172.21.15.31:0/1265492559 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:13:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb24effd700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb250103cd0 0x7fb2501991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb24ffff700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250104670 0x7fb2501a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:02.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb24f7fe700 1 -- 172.21.15.31:0/1265492559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb250103cd0 msgr2=0x7fb2501991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:02.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb24f7fe700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb250103cd0 0x7fb2501991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb24f7fe700 1 -- 172.21.15.31:0/1265492559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250104670 msgr2=0x7fb2501a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb24f7fe700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250104670 0x7fb2501a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb24f7fe700 1 -- 172.21.15.31:0/1265492559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2501a0870 con 0x7fb250102ad0 2024-04-25T13:13:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.610+0000 7fb24f7fe700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb250102ad0 0x7fb250198c70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb240006010 tx=0x7fb2400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.611+0000 7fb24cff9700 1 -- 172.21.15.31:0/1265492559 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240027070 con 0x7fb250102ad0 2024-04-25T13:13:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.611+0000 7fb24cff9700 1 -- 172.21.15.31:0/1265492559 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2400040e0 con 0x7fb250102ad0 2024-04-25T13:13:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.611+0000 7fb25614b700 1 -- 172.21.15.31:0/1265492559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2501a0b00 con 0x7fb250102ad0 2024-04-25T13:13:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.611+0000 7fb24cff9700 1 -- 172.21.15.31:0/1265492559 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24000f460 con 0x7fb250102ad0 2024-04-25T13:13:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.611+0000 7fb25614b700 1 -- 172.21.15.31:0/1265492559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2501a0fb0 con 0x7fb250102ad0 2024-04-25T13:13:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.612+0000 7fb25614b700 1 -- 172.21.15.31:0/1265492559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb250069730 con 0x7fb250102ad0 2024-04-25T13:13:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.613+0000 7fb24cff9700 1 -- 172.21.15.31:0/1265492559 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fb24002a070 con 0x7fb250102ad0 2024-04-25T13:13:02.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.613+0000 7fb24cff9700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2380a3c30 0x7fb2380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:02.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.614+0000 7fb24effd700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2380a3c30 0x7fb2380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:02.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.614+0000 7fb24cff9700 1 -- 172.21.15.31:0/1265492559 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7fb2400d0480 con 0x7fb250102ad0 2024-04-25T13:13:02.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.614+0000 7fb24effd700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2380a3c30 0x7fb2380a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb244005950 tx=0x7fb24400a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:02.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.618+0000 7fb24cff9700 1 -- 172.21.15.31:0/1265492559 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb240031070 con 0x7fb250102ad0 2024-04-25T13:13:02.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:02 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi136 2024-04-25T13:13:02.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:02 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi136 to 17088M 2024-04-25T13:13:02.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:02 smithi066 ceph-mon[33137]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:13:02.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:13:02.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:02.835+0000 7fb25614b700 1 -- 172.21.15.31:0/1265492559 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb250062e80 con 0x7fb2380a3c30 2024-04-25T13:13:03.460 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41989]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:13:03.461 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41989]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.461 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41989]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.461 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41989]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:03.461 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 ceph-mon[33038]: pgmap v84: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:03.461 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:13:03.461 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 ceph-mon[33038]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:13:03.461 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:13:03.461 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:13:03.462 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:13:03.462 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:03.483 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48745]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:13:03.484 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48745]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.484 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48745]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.484 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48745]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:03.484 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48762]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:13:03.484 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48762]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.484 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48762]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.484 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48762]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:03.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 ceph-mon[29457]: pgmap v84: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:03.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:13:03.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 ceph-mon[29457]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:13:03.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:13:03.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:13:03.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:13:03.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:03.581 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 ceph-mon[33137]: pgmap v84: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:03.581 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:13:03.581 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 ceph-mon[33137]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:13:03.581 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:13:03.582 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:13:03.582 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:13:03.582 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:03.582 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41745]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:13:03.582 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41745]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.582 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41745]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.582 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41745]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:03.582 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41748]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:13:03.582 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41748]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.582 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41748]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.583 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41748]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:03.719 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41992]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:13:03.719 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41992]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.719 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41992]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.719 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41992]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:03.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41995]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:13:03.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41995]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41995]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41995]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:03.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41998]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:13:03.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41998]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41998]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:03 smithi136 sudo[41998]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:03.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48793]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:13:03.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48793]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48793]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48793]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:03.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48796]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:13:03.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48796]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48796]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:03 smithi031 sudo[48796]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:03.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41751]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:13:03.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41751]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41751]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41751]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:03.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41754]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:13:03.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41754]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:13:03.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41754]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:13:03.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:03 smithi066 sudo[41754]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:13:04.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:04.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:13:04.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:13:04.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:04.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:04.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:04.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:13:04.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:13:04.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:13:04.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:13:04.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:13:04.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:04 smithi136 ceph-mon[33038]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:04.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:04 smithi031 ceph-mon[29457]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:13:04.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:04.510+0000 7fb24cff9700 1 -- 172.21.15.31:0/1265492559 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb24009a280 con 0x7fb250102ad0 2024-04-25T13:13:04.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:04.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:13:04.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:13:04.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:04.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:04.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:04.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:13:04.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:13:04.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:13:04.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:13:04.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:13:04.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:13:04.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:13:04.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:13:04.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:04 smithi066 ceph-mon[33137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:13:05.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:05 smithi136 ceph-mon[33038]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:05.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:05 smithi136 ceph-mon[33038]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T13:13:05.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:05 smithi136 ceph-mon[33038]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:13:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:05 smithi031 ceph-mon[29457]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:05 smithi031 ceph-mon[29457]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T13:13:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:05 smithi031 ceph-mon[29457]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:13:05.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:05 smithi066 ceph-mon[33137]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:05.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:05 smithi066 ceph-mon[33137]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T13:13:05.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:05 smithi066 ceph-mon[33137]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:13:06.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:06 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/4063498570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dfa360c-0171-4df2-ad03-690209f41d77"}]: dispatch 2024-04-25T13:13:06.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:06 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/4063498570' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dfa360c-0171-4df2-ad03-690209f41d77"}]': finished 2024-04-25T13:13:06.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:06 smithi136 ceph-mon[33038]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:13:06.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:06 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:06.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:06 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/4063498570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dfa360c-0171-4df2-ad03-690209f41d77"}]: dispatch 2024-04-25T13:13:06.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:06 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/4063498570' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dfa360c-0171-4df2-ad03-690209f41d77"}]': finished 2024-04-25T13:13:06.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:06 smithi031 ceph-mon[29457]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:13:06.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:06 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:06.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:06 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/4063498570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9dfa360c-0171-4df2-ad03-690209f41d77"}]: dispatch 2024-04-25T13:13:06.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:06 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/4063498570' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9dfa360c-0171-4df2-ad03-690209f41d77"}]': finished 2024-04-25T13:13:06.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:06 smithi066 ceph-mon[33137]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:13:06.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:06 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:07.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:07 smithi136 ceph-mon[33038]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:07.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:07 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1087718886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:13:07.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:07 smithi031 ceph-mon[29457]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:07.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:07 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1087718886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:13:07.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:07 smithi066 ceph-mon[33137]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:07.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:07 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1087718886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:13:09.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:09 smithi136 ceph-mon[33038]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:09.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:09 smithi031 ceph-mon[29457]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:09.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:09 smithi066 ceph-mon[33137]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:11.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:11 smithi031 ceph-mon[29457]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:11.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:11 smithi066 ceph-mon[33137]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:11.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:11 smithi136 ceph-mon[33038]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:13.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:13 smithi031 ceph-mon[29457]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:13.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:13 smithi066 ceph-mon[33137]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:13.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:13 smithi136 ceph-mon[33038]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:14.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:13:14.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:14.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:13:14.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:14.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:13:14.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:15.338 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:15 smithi031 ceph-mon[29457]: Deploying daemon osd.3 on smithi031 2024-04-25T13:13:15.338 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:15 smithi031 ceph-mon[29457]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:15.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:15 smithi066 ceph-mon[33137]: Deploying daemon osd.3 on smithi031 2024-04-25T13:13:15.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:15 smithi066 ceph-mon[33137]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:15.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:15 smithi136 ceph-mon[33038]: Deploying daemon osd.3 on smithi031 2024-04-25T13:13:15.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:15 smithi136 ceph-mon[33038]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:17.419 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:17 smithi031 ceph-mon[29457]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:17.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:17 smithi066 ceph-mon[33137]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:17.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:17 smithi136 ceph-mon[33038]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:19.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:19 smithi031 ceph-mon[29457]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:19.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:19 smithi066 ceph-mon[33137]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:19.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:19 smithi136 ceph-mon[33038]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:20.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:20 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:20.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:20 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:20.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:20 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:20.702 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:20 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:20.703 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:20 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:20.703 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:20 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:20.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:20 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:20.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:20 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:20.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:20 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:21.613 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:21 smithi031 ceph-mon[29457]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:21.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:21 smithi066 ceph-mon[33137]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:21.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:21 smithi136 ceph-mon[33038]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:22.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:22 smithi031 ceph-mon[29457]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:23.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.147+0000 7fb24cff9700 1 -- 172.21.15.31:0/1265492559 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb250062e80 con 0x7fb2380a3c30 2024-04-25T13:13:23.148 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 3 on host 'smithi031' 2024-04-25T13:13:23.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 -- 172.21.15.31:0/1265492559 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2380a3c30 msgr2=0x7fb2380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2380a3c30 0x7fb2380a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb244005950 tx=0x7fb24400a300 comp rx=0 tx=0).stop 2024-04-25T13:13:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 -- 172.21.15.31:0/1265492559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb250102ad0 msgr2=0x7fb250198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb250102ad0 0x7fb250198c70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb240006010 tx=0x7fb2400047b0 comp rx=0 tx=0).stop 2024-04-25T13:13:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 -- 172.21.15.31:0/1265492559 shutdown_connections 2024-04-25T13:13:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb250104670 0x7fb2501a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb250102ad0 0x7fb250198c70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2380a3c30 0x7fb2380a60f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 --2- 172.21.15.31:0/1265492559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb250103cd0 0x7fb2501991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 -- 172.21.15.31:0/1265492559 >> 172.21.15.31:0/1265492559 conn(0x7fb2500fe090 msgr2=0x7fb250107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 -- 172.21.15.31:0/1265492559 shutdown_connections 2024-04-25T13:13:23.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:13:23.150+0000 7fb25614b700 1 -- 172.21.15.31:0/1265492559 wait complete. 2024-04-25T13:13:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:22 smithi066 ceph-mon[33137]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:22 smithi136 ceph-mon[33038]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:23.774 DEBUG:teuthology.orchestra.run.smithi031:osd.3> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T13:13:23.777 INFO:tasks.cephadm:Deploying osd.4 on smithi066 with /dev/vg_nvme/lv_3... 2024-04-25T13:13:23.777 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:13:24.032 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:13:24.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:24.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:24.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:24.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:24 smithi031 ceph-mon[29457]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:13:24.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:24 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:24.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:24 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:24.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:24 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:24.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:24 smithi066 ceph-mon[33137]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:13:24.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:24 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:24.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:24 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:24.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:24 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:24.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:24 smithi136 ceph-mon[33038]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:13:25.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:25 smithi031 ceph-mon[29457]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:25 smithi031 ceph-mon[29457]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:13:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:25 smithi031 ceph-mon[29457]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:13:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:25 smithi031 ceph-mon[29457]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:13:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:25 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:25.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:25 smithi066 ceph-mon[33137]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:25 smithi066 ceph-mon[33137]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:13:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:25 smithi066 ceph-mon[33137]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:13:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:25 smithi066 ceph-mon[33137]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:13:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:25 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:25.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:25 smithi136 ceph-mon[33038]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:25.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:25 smithi136 ceph-mon[33038]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:13:25.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:25 smithi136 ceph-mon[33038]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:13:25.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:25 smithi136 ceph-mon[33038]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:13:25.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:25 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:26.102 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:13:26.103 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:13:26.103 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-04-25T13:13:26.103 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-04-25T13:13:26.103 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111975 s, 936 MB/s 2024-04-25T13:13:26.103 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-04-25T13:13:26.405 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:26 smithi031 ceph-mon[29457]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:13:26.405 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:26 smithi031 ceph-mon[29457]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T13:13:26.405 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:26.405 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:26.406 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:13:26 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3[50953]: 2024-04-25T13:13:26.143+0000 7f6b79d95700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T13:13:26.406 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:13:26 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3[50953]: 2024-04-25T13:13:26.174+0000 7f6b71b80700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:13:26.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:26 smithi066 ceph-mon[33137]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:13:26.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:26 smithi066 ceph-mon[33137]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T13:13:26.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:26.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:26.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:26 smithi136 ceph-mon[33038]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:13:26.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:26 smithi136 ceph-mon[33038]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T13:13:26.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:26.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:26.713 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_3 2024-04-25T13:13:26.967 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:27.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:27.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:27.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:27.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:27.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:27.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:13:27.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958]' entity='osd.3' 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:28.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:28 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi031 2024-04-25T13:13:28.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:28 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi031 to 8546M 2024-04-25T13:13:28.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:28 smithi066 ceph-mon[33137]: osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958] boot 2024-04-25T13:13:28.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:28 smithi066 ceph-mon[33137]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:13:28.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:28 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:28.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:28 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi031 2024-04-25T13:13:28.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:28 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi031 to 8546M 2024-04-25T13:13:28.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:28 smithi136 ceph-mon[33038]: osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958] boot 2024-04-25T13:13:28.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:28 smithi136 ceph-mon[33038]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:13:28.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:28 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:28.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:28 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi031 2024-04-25T13:13:28.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:28 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi031 to 8546M 2024-04-25T13:13:28.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:28 smithi031 ceph-mon[29457]: osd.3 [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958] boot 2024-04-25T13:13:28.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:28 smithi031 ceph-mon[29457]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:13:28.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:28 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:13:28.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.662+0000 7fb4c9f16700 1 -- 172.21.15.66:0/3356260191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c4075420 msgr2=0x7fb4c40758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.662+0000 7fb4c9f16700 1 --2- 172.21.15.66:0/3356260191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c4075420 0x7fb4c40758a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009990 tx=0x7fb4ac009ca0 comp rx=0 tx=0).stop 2024-04-25T13:13:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.662+0000 7fb4c9f16700 1 -- 172.21.15.66:0/3356260191 shutdown_connections 2024-04-25T13:13:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.662+0000 7fb4c9f16700 1 --2- 172.21.15.66:0/3356260191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4c410a9c0 0x7fb4c410ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.662+0000 7fb4c9f16700 1 --2- 172.21.15.66:0/3356260191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4c40769f0 0x7fb4c4074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.662+0000 7fb4c9f16700 1 --2- 172.21.15.66:0/3356260191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c4075420 0x7fb4c40758a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.662+0000 7fb4c9f16700 1 -- 172.21.15.66:0/3356260191 >> 172.21.15.66:0/3356260191 conn(0x7fb4c40fe060 msgr2=0x7fb4c41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.662+0000 7fb4c9f16700 1 -- 172.21.15.66:0/3356260191 shutdown_connections 2024-04-25T13:13:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.662+0000 7fb4c9f16700 1 -- 172.21.15.66:0/3356260191 wait complete. 2024-04-25T13:13:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.663+0000 7fb4c9f16700 1 Processor -- start 2024-04-25T13:13:28.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.663+0000 7fb4c9f16700 1 -- start start 2024-04-25T13:13:28.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.663+0000 7fb4c9f16700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4c4075420 0x7fb4c419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:28.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.663+0000 7fb4c9f16700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4c40769f0 0x7fb4c419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:28.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.663+0000 7fb4c9f16700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c410a9c0 0x7fb4c41a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:28.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.663+0000 7fb4c9f16700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4110740 con 0x7fb4c40769f0 2024-04-25T13:13:28.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.663+0000 7fb4c9f16700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41105c0 con 0x7fb4c410a9c0 2024-04-25T13:13:28.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.663+0000 7fb4c9f16700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41108c0 con 0x7fb4c4075420 2024-04-25T13:13:28.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.663+0000 7fb4c3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c410a9c0 0x7fb4c41a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:28.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.663+0000 7fb4c3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c410a9c0 0x7fb4c41a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44746/0 (socket says 172.21.15.66:44746) 2024-04-25T13:13:28.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.663+0000 7fb4c3fff700 1 -- 172.21.15.66:0/933866631 learned_addr learned my addr 172.21.15.66:0/933866631 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-25T13:13:28.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c37fe700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4c4075420 0x7fb4c419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:28.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c2ffd700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4c40769f0 0x7fb4c419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:28.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c3fff700 1 -- 172.21.15.66:0/933866631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4c4075420 msgr2=0x7fb4c419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:28.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c3fff700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4c4075420 0x7fb4c419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:28.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c3fff700 1 -- 172.21.15.66:0/933866631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4c40769f0 msgr2=0x7fb4c419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:28.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c3fff700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4c40769f0 0x7fb4c419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:28.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c3fff700 1 -- 172.21.15.66:0/933866631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c41a2a10 con 0x7fb4c410a9c0 2024-04-25T13:13:28.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c3fff700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c410a9c0 0x7fb4c41a2370 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e940 tx=0x7fb4b400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:28.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c0ff9700 1 -- 172.21.15.66:0/933866631 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400cbd0 con 0x7fb4c410a9c0 2024-04-25T13:13:28.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c0ff9700 1 -- 172.21.15.66:0/933866631 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b400cd70 con 0x7fb4c410a9c0 2024-04-25T13:13:28.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c9f16700 1 -- 172.21.15.66:0/933866631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c41a2d00 con 0x7fb4c410a9c0 2024-04-25T13:13:28.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c0ff9700 1 -- 172.21.15.66:0/933866631 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4005680 con 0x7fb4c410a9c0 2024-04-25T13:13:28.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.664+0000 7fb4c9f16700 1 -- 172.21.15.66:0/933866631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c41a31e0 con 0x7fb4c410a9c0 2024-04-25T13:13:28.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.665+0000 7fb4c9f16700 1 -- 172.21.15.66:0/933866631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c4069730 con 0x7fb4c410a9c0 2024-04-25T13:13:28.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.666+0000 7fb4c0ff9700 1 -- 172.21.15.66:0/933866631 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb4b4004750 con 0x7fb4c410a9c0 2024-04-25T13:13:28.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.667+0000 7fb4c0ff9700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4b00a3c40 0x7fb4b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:28.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.667+0000 7fb4c0ff9700 1 -- 172.21.15.66:0/933866631 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7fb4b40cf3b0 con 0x7fb4c410a9c0 2024-04-25T13:13:28.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.667+0000 7fb4c37fe700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4b00a3c40 0x7fb4b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:28.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.668+0000 7fb4c37fe700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4b00a3c40 0x7fb4b00a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009d90 tx=0x7fb4bc009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:28.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.672+0000 7fb4c0ff9700 1 -- 172.21.15.66:0/933866631 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4b409cf90 con 0x7fb4c410a9c0 2024-04-25T13:13:28.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:28.890+0000 7fb4c9f16700 1 -- 172.21.15.66:0/933866631 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb4c4062e80 con 0x7fb4b00a3c40 2024-04-25T13:13:29.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:29 smithi136 ceph-mon[33038]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:29.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:29 smithi136 ceph-mon[33038]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:13:29.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:29 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:13:29.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:29 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:13:29.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:29 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:29.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:29 smithi031 ceph-mon[29457]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:29.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:29 smithi031 ceph-mon[29457]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:13:29.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:29 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:13:29.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:29 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:13:29.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:29 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:29.528 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:29 smithi066 ceph-mon[33137]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:13:29.528 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:29 smithi066 ceph-mon[33137]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:13:29.528 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:29 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:13:29.529 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:29 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:13:29.529 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:29 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:30.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:30 smithi136 ceph-mon[33038]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:30.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:30 smithi136 ceph-mon[33038]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T13:13:30.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:30 smithi136 ceph-mon[33038]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T13:13:30.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:30 smithi031 ceph-mon[29457]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:30.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:30 smithi031 ceph-mon[29457]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T13:13:30.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:30 smithi031 ceph-mon[29457]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T13:13:30.516 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:30 smithi066 ceph-mon[33137]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:30.516 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:30 smithi066 ceph-mon[33137]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T13:13:30.516 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:30 smithi066 ceph-mon[33137]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T13:13:31.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:31 smithi136 ceph-mon[33038]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:31.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:31 smithi136 ceph-mon[33038]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T13:13:31.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:31 smithi031 ceph-mon[29457]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:31.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:31 smithi031 ceph-mon[29457]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T13:13:31.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:31 smithi066 ceph-mon[33137]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:31.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:31 smithi066 ceph-mon[33137]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T13:13:32.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:32 smithi066 ceph-mon[33137]: from='client.? 172.21.15.66:0/2048223173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108"}]: dispatch 2024-04-25T13:13:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:32 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108"}]: dispatch 2024-04-25T13:13:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:32 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108"}]': finished 2024-04-25T13:13:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:32 smithi066 ceph-mon[33137]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T13:13:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:32.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:32 smithi136 ceph-mon[33038]: from='client.? 172.21.15.66:0/2048223173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108"}]: dispatch 2024-04-25T13:13:32.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:32 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108"}]: dispatch 2024-04-25T13:13:32.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:32 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108"}]': finished 2024-04-25T13:13:32.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:32 smithi136 ceph-mon[33038]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T13:13:32.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:32.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:32 smithi031 ceph-mon[29457]: from='client.? 172.21.15.66:0/2048223173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108"}]: dispatch 2024-04-25T13:13:32.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:32 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108"}]: dispatch 2024-04-25T13:13:32.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:32 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108"}]': finished 2024-04-25T13:13:32.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:32 smithi031 ceph-mon[29457]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T13:13:32.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:33.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:33 smithi136 ceph-mon[33038]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 161 KiB/s, 0 objects/s recovering 2024-04-25T13:13:33.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:33 smithi136 ceph-mon[33038]: from='client.? 172.21.15.66:0/3808026358' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:13:33.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:33 smithi031 ceph-mon[29457]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 161 KiB/s, 0 objects/s recovering 2024-04-25T13:13:33.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:33 smithi031 ceph-mon[29457]: from='client.? 172.21.15.66:0/3808026358' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:13:33.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:33 smithi066 ceph-mon[33137]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 161 KiB/s, 0 objects/s recovering 2024-04-25T13:13:33.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:33 smithi066 ceph-mon[33137]: from='client.? 172.21.15.66:0/3808026358' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:13:35.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:35 smithi136 ceph-mon[33038]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-25T13:13:35.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:35 smithi031 ceph-mon[29457]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-25T13:13:35.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:35 smithi066 ceph-mon[33137]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-25T13:13:37.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:37 smithi136 ceph-mon[33038]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:13:37.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:37 smithi031 ceph-mon[29457]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:13:37.488 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:37 smithi066 ceph-mon[33137]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:13:39.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:39 smithi066 ceph-mon[33137]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:13:39.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:39 smithi136 ceph-mon[33038]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:13:39.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:39 smithi031 ceph-mon[29457]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:13:40.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:13:40.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:40.323 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:40 smithi066 ceph-mon[33137]: Deploying daemon osd.4 on smithi066 2024-04-25T13:13:40.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:13:40.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:40.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:40 smithi136 ceph-mon[33038]: Deploying daemon osd.4 on smithi066 2024-04-25T13:13:40.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:13:40.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:40.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:40 smithi031 ceph-mon[29457]: Deploying daemon osd.4 on smithi066 2024-04-25T13:13:41.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:41 smithi066 ceph-mon[33137]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:13:41.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:41 smithi136 ceph-mon[33038]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:13:41.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:41 smithi031 ceph-mon[29457]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:13:43.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:43 smithi136 ceph-mon[33038]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:43.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:43 smithi031 ceph-mon[29457]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:43.625 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:43 smithi066 ceph-mon[33137]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:45.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:45 smithi136 ceph-mon[33038]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:45.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:45 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:45.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:45 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:45.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:45 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:45.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:45 smithi031 ceph-mon[29457]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:45.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:45 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:45.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:45 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:45.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:45 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:45.510 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:45 smithi066 ceph-mon[33137]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:45.511 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:45 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:45.511 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:45 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:45.511 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:45 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:47.456 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:47 smithi066 ceph-mon[33137]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:47.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:47 smithi136 ceph-mon[33038]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:47.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:47 smithi031 ceph-mon[29457]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:48.611 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 4 on host 'smithi066' 2024-04-25T13:13:48.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.611+0000 7fb4c0ff9700 1 -- 172.21.15.66:0/933866631 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb4c4062e80 con 0x7fb4b00a3c40 2024-04-25T13:13:48.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 -- 172.21.15.66:0/933866631 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4b00a3c40 msgr2=0x7fb4b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:48.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4b00a3c40 0x7fb4b00a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009d90 tx=0x7fb4bc009040 comp rx=0 tx=0).stop 2024-04-25T13:13:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 -- 172.21.15.66:0/933866631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c410a9c0 msgr2=0x7fb4c41a2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c410a9c0 0x7fb4c41a2370 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e940 tx=0x7fb4b400ed00 comp rx=0 tx=0).stop 2024-04-25T13:13:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 -- 172.21.15.66:0/933866631 shutdown_connections 2024-04-25T13:13:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4b00a3c40 0x7fb4b00a6100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4c4075420 0x7fb4c419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4c410a9c0 0x7fb4c41a2370 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 --2- 172.21.15.66:0/933866631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4c40769f0 0x7fb4c419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 -- 172.21.15.66:0/933866631 >> 172.21.15.66:0/933866631 conn(0x7fb4c40fe060 msgr2=0x7fb4c4106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 -- 172.21.15.66:0/933866631 shutdown_connections 2024-04-25T13:13:48.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:13:48.613+0000 7fb4c9f16700 1 -- 172.21.15.66:0/933866631 wait complete. 2024-04-25T13:13:48.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:48.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:48.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:48 smithi066 ceph-mon[33137]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:48.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:48.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:48 smithi136 ceph-mon[33038]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:48.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:48.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:48.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:48 smithi031 ceph-mon[29457]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:48.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:48.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:13:49.287 DEBUG:teuthology.orchestra.run.smithi066:osd.4> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T13:13:49.290 INFO:tasks.cephadm:Deploying osd.5 on smithi136 with /dev/vg_nvme/lv_3... 2024-04-25T13:13:49.290 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:13:49.538 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:13:49.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:49 smithi066 ceph-mon[33137]: from='osd.4 [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:13:49.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:49 smithi066 ceph-mon[33137]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:13:49.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:49 smithi136 ceph-mon[33038]: from='osd.4 [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:13:49.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:49 smithi136 ceph-mon[33038]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:13:49.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:49 smithi031 ceph-mon[29457]: from='osd.4 [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:13:49.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:49 smithi031 ceph-mon[29457]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:13:50.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:50 smithi066 ceph-mon[33137]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:13:50.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:50 smithi066 ceph-mon[33137]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T13:13:50.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:50 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:50.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:50 smithi066 ceph-mon[33137]: from='osd.4 [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:13:50.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:50 smithi066 ceph-mon[33137]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:13:50.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:50 smithi066 ceph-mon[33137]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:50.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:50 smithi136 ceph-mon[33038]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:13:50.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:50 smithi136 ceph-mon[33038]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T13:13:50.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:50 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:50.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:50 smithi136 ceph-mon[33038]: from='osd.4 [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:13:50.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:50 smithi136 ceph-mon[33038]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:13:50.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:50 smithi136 ceph-mon[33038]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:50.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:50 smithi031 ceph-mon[29457]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:13:50.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:50 smithi031 ceph-mon[29457]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T13:13:50.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:50 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:50.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:50 smithi031 ceph-mon[29457]: from='osd.4 [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:13:50.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:50 smithi031 ceph-mon[29457]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:13:50.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:50 smithi031 ceph-mon[29457]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:51.567 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:13:51.567 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:13:51.567 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-04-25T13:13:51.567 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-04-25T13:13:51.567 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00940073 s, 1.1 GB/s 2024-04-25T13:13:51.568 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-04-25T13:13:51.870 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:51 smithi136 ceph-mon[33038]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:13:51.870 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:51 smithi136 ceph-mon[33038]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T13:13:51.870 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:51 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:51.870 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:51 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:51.871 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:51 smithi136 ceph-mon[33038]: from='osd.4 ' entity='osd.4' 2024-04-25T13:13:51.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:51 smithi066 ceph-mon[33137]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:13:51.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:51 smithi066 ceph-mon[33137]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T13:13:51.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:51 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:51.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:51 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:51.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:51 smithi066 ceph-mon[33137]: from='osd.4 ' entity='osd.4' 2024-04-25T13:13:51.909 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:13:51 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4[44554]: 2024-04-25T13:13:51.620+0000 7f72e721c700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T13:13:51.909 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:13:51 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4[44554]: 2024-04-25T13:13:51.716+0000 7f72df007700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:13:51.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:51 smithi031 ceph-mon[29457]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:13:51.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:51 smithi031 ceph-mon[29457]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T13:13:51.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:51 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:51.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:51 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:51.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:51 smithi031 ceph-mon[29457]: from='osd.4 ' entity='osd.4' 2024-04-25T13:13:52.170 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_3 2024-04-25T13:13:52.413 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:13:52.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:13:52.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:13:52.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:52.793 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:52.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi066 2024-04-25T13:13:52.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:52.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:52.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:52.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:52.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi066 to 8544M 2024-04-25T13:13:52.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:52.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:52.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:52.794 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:52.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi066 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi066 to 8544M 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:52.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:13:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:13:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:13:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi066 2024-04-25T13:13:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:13:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi066 to 8544M 2024-04-25T13:13:53.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:53.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:53.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:13:53.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:13:53.965 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:53 smithi136 ceph-mon[33038]: osd.4 [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233] boot 2024-04-25T13:13:53.966 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:53 smithi136 ceph-mon[33038]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T13:13:53.966 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:53.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.977+0000 7faf9f677700 1 -- 172.21.15.136:0/1831342037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf98105eb0 msgr2=0x7faf981082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.977+0000 7faf9f677700 1 --2- 172.21.15.136:0/1831342037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf98105eb0 0x7faf981082a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faf94009990 tx=0x7faf94009ca0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.977+0000 7faf9f677700 1 -- 172.21.15.136:0/1831342037 shutdown_connections 2024-04-25T13:13:53.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.977+0000 7faf9f677700 1 --2- 172.21.15.136:0/1831342037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf981087e0 0x7faf9810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.977+0000 7faf9f677700 1 --2- 172.21.15.136:0/1831342037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98103580 0x7faf98105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.977+0000 7faf9f677700 1 --2- 172.21.15.136:0/1831342037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf98105eb0 0x7faf981082a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.977+0000 7faf9f677700 1 -- 172.21.15.136:0/1831342037 >> 172.21.15.136:0/1831342037 conn(0x7faf980fcf30 msgr2=0x7faf980ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:13:53.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.977+0000 7faf9f677700 1 -- 172.21.15.136:0/1831342037 shutdown_connections 2024-04-25T13:13:53.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.977+0000 7faf9f677700 1 -- 172.21.15.136:0/1831342037 wait complete. 2024-04-25T13:13:53.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9f677700 1 Processor -- start 2024-04-25T13:13:53.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9f677700 1 -- start start 2024-04-25T13:13:53.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9f677700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf98103580 0x7faf98102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9f677700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98105eb0 0x7faf98101530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9f677700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf981087e0 0x7faf98101a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9f677700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faf9810e590 con 0x7faf98105eb0 2024-04-25T13:13:53.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9f677700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faf9810e410 con 0x7faf981087e0 2024-04-25T13:13:53.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9f677700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf9810e710 con 0x7faf98103580 2024-04-25T13:13:53.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9d413700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf98103580 0x7faf98102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9d413700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf98103580 0x7faf98102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:39206/0 (socket says 172.21.15.136:39206) 2024-04-25T13:13:53.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9d413700 1 -- 172.21.15.136:0/3398287664 learned_addr learned my addr 172.21.15.136:0/3398287664 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9dc14700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf981087e0 0x7faf98101a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9d413700 1 -- 172.21.15.136:0/3398287664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf981087e0 msgr2=0x7faf98101a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9d413700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf981087e0 0x7faf98101a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9d413700 1 -- 172.21.15.136:0/3398287664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98105eb0 msgr2=0x7faf98101530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9cc12700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98105eb0 0x7faf98101530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9d413700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98105eb0 0x7faf98101530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9d413700 1 -- 172.21.15.136:0/3398287664 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf98102320 con 0x7faf98103580 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9cc12700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98105eb0 0x7faf98101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.978+0000 7faf9d413700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf98103580 0x7faf98102ee0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c920 tx=0x7faf8800cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.979+0000 7faf8e7fc700 1 -- 172.21.15.136:0/3398287664 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88007b00 con 0x7faf98103580 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.979+0000 7faf8e7fc700 1 -- 172.21.15.136:0/3398287664 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf88004ce0 con 0x7faf98103580 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.979+0000 7faf9f677700 1 -- 172.21.15.136:0/3398287664 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf981a2dc0 con 0x7faf98103580 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.979+0000 7faf8e7fc700 1 -- 172.21.15.136:0/3398287664 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88005650 con 0x7faf98103580 2024-04-25T13:13:53.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.979+0000 7faf9f677700 1 -- 172.21.15.136:0/3398287664 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf981a3290 con 0x7faf98103580 2024-04-25T13:13:53.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.980+0000 7faf9f677700 1 -- 172.21.15.136:0/3398287664 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf98069730 con 0x7faf98103580 2024-04-25T13:13:53.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.981+0000 7faf8e7fc700 1 -- 172.21.15.136:0/3398287664 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faf88004750 con 0x7faf98103580 2024-04-25T13:13:53.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.981+0000 7faf8e7fc700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faf840a3c40 0x7faf840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:13:53.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.981+0000 7faf8e7fc700 1 -- 172.21.15.136:0/3398287664 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4101+0+0 (secure 0 0 0) 0x7faf880cefe0 con 0x7faf98103580 2024-04-25T13:13:53.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.982+0000 7faf9cc12700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faf840a3c40 0x7faf840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:13:53.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.982+0000 7faf9cc12700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faf840a3c40 0x7faf840a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf94006010 tx=0x7faf940095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:13:53.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:53 smithi031 ceph-mon[29457]: osd.4 [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233] boot 2024-04-25T13:13:53.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:53 smithi031 ceph-mon[29457]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T13:13:53.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:53.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:53.986+0000 7faf8e7fc700 1 -- 172.21.15.136:0/3398287664 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf88014030 con 0x7faf98103580 2024-04-25T13:13:54.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:53 smithi066 ceph-mon[33137]: osd.4 [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233] boot 2024-04-25T13:13:54.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:53 smithi066 ceph-mon[33137]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T13:13:54.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:13:54.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:13:54.190+0000 7faf9f677700 1 -- 172.21.15.136:0/3398287664 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7faf98062e80 con 0x7faf840a3c40 2024-04-25T13:13:55.051 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:54 smithi136 ceph-mon[33038]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T13:13:55.051 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:54 smithi136 ceph-mon[33038]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:13:55.051 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:54 smithi136 ceph-mon[33038]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:55.051 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:13:55.051 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:13:55.051 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:55.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:54 smithi066 ceph-mon[33137]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T13:13:55.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:54 smithi066 ceph-mon[33137]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:13:55.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:54 smithi066 ceph-mon[33137]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:55.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:13:55.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:13:55.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:55.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:54 smithi031 ceph-mon[29457]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T13:13:55.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:54 smithi031 ceph-mon[29457]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:13:55.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:54 smithi031 ceph-mon[29457]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:13:55.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:13:55.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:13:55.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:13:57.149 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:56 smithi031 ceph-mon[29457]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:13:57.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:56 smithi066 ceph-mon[33137]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:13:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:56 smithi136 ceph-mon[33038]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:13:58.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:57 smithi066 ceph-mon[33137]: from='client.? 172.21.15.136:0/175649778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e70ec561-45b2-4678-9d4a-f94c95a3a29a"}]: dispatch 2024-04-25T13:13:58.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:57 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e70ec561-45b2-4678-9d4a-f94c95a3a29a"}]: dispatch 2024-04-25T13:13:58.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:57 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e70ec561-45b2-4678-9d4a-f94c95a3a29a"}]': finished 2024-04-25T13:13:58.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:57 smithi066 ceph-mon[33137]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:13:58.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:13:58.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:57 smithi066 ceph-mon[33137]: from='client.? 172.21.15.136:0/3653054562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:13:58.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:57 smithi136 ceph-mon[33038]: from='client.? 172.21.15.136:0/175649778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e70ec561-45b2-4678-9d4a-f94c95a3a29a"}]: dispatch 2024-04-25T13:13:58.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:57 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e70ec561-45b2-4678-9d4a-f94c95a3a29a"}]: dispatch 2024-04-25T13:13:58.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:57 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e70ec561-45b2-4678-9d4a-f94c95a3a29a"}]': finished 2024-04-25T13:13:58.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:57 smithi136 ceph-mon[33038]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:13:58.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:13:58.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:57 smithi136 ceph-mon[33038]: from='client.? 172.21.15.136:0/3653054562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:13:58.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:57 smithi031 ceph-mon[29457]: from='client.? 172.21.15.136:0/175649778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e70ec561-45b2-4678-9d4a-f94c95a3a29a"}]: dispatch 2024-04-25T13:13:58.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:57 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e70ec561-45b2-4678-9d4a-f94c95a3a29a"}]: dispatch 2024-04-25T13:13:58.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:57 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e70ec561-45b2-4678-9d4a-f94c95a3a29a"}]': finished 2024-04-25T13:13:58.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:57 smithi031 ceph-mon[29457]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:13:58.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:13:58.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:57 smithi031 ceph-mon[29457]: from='client.? 172.21.15.136:0/3653054562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:13:59.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:13:58 smithi136 ceph-mon[33038]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:13:59.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:13:58 smithi031 ceph-mon[29457]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:13:59.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:13:58 smithi066 ceph-mon[33137]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:01.161 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:00 smithi136 ceph-mon[33038]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:01.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:00 smithi031 ceph-mon[29457]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:01.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:00 smithi066 ceph-mon[33137]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:03.200 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:02 smithi136 ceph-mon[33038]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:03.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:02 smithi031 ceph-mon[29457]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:03.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:02 smithi066 ceph-mon[33137]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:05.193 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:04 smithi136 ceph-mon[33038]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:05.193 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:14:05.193 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:05.193 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:04 smithi136 ceph-mon[33038]: Deploying daemon osd.5 on smithi136 2024-04-25T13:14:05.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:04 smithi031 ceph-mon[29457]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:05.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:14:05.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:05.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:04 smithi031 ceph-mon[29457]: Deploying daemon osd.5 on smithi136 2024-04-25T13:14:05.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:04 smithi066 ceph-mon[33137]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:05.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:14:05.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:05.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:04 smithi066 ceph-mon[33137]: Deploying daemon osd.5 on smithi136 2024-04-25T13:14:07.185 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:06 smithi136 ceph-mon[33038]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:07.224 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:06 smithi031 ceph-mon[29457]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:07.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:06 smithi066 ceph-mon[33137]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:09.196 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:08 smithi136 ceph-mon[33038]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:09.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:08 smithi031 ceph-mon[29457]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:09.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:08 smithi066 ceph-mon[33137]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:10 smithi066 ceph-mon[33137]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:10 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:10 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:10 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:11.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:10 smithi136 ceph-mon[33038]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:11.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:10 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:11.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:10 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:11.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:10 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:11.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:10 smithi031 ceph-mon[29457]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:11.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:11.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:11.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:13.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:12 smithi066 ceph-mon[33137]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:13.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:12 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:13.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:12 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:13.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:12 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:13.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:12 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:13.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:12 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:13.193 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:12 smithi136 ceph-mon[33038]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:13.194 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:12 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:13.194 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:12 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:13.194 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:12 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:13.194 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:12 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:13.194 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:12 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:13.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:12 smithi031 ceph-mon[29457]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:13.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:13.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:13.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:13.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:13.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:12 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:13.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.615+0000 7faf8e7fc700 1 -- 172.21.15.136:0/3398287664 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faf98062e80 con 0x7faf840a3c40 2024-04-25T13:14:13.616 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 5 on host 'smithi136' 2024-04-25T13:14:13.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 -- 172.21.15.136:0/3398287664 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faf840a3c40 msgr2=0x7faf840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:13.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faf840a3c40 0x7faf840a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf94006010 tx=0x7faf940095f0 comp rx=0 tx=0).stop 2024-04-25T13:14:13.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 -- 172.21.15.136:0/3398287664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf98103580 msgr2=0x7faf98102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:13.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf98103580 0x7faf98102ee0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c920 tx=0x7faf8800cce0 comp rx=0 tx=0).stop 2024-04-25T13:14:13.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 -- 172.21.15.136:0/3398287664 shutdown_connections 2024-04-25T13:14:13.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faf840a3c40 0x7faf840a6100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:13.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf98103580 0x7faf98102ee0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:13.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faf981087e0 0x7faf98101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:13.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 --2- 172.21.15.136:0/3398287664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98105eb0 0x7faf98101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:13.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 -- 172.21.15.136:0/3398287664 >> 172.21.15.136:0/3398287664 conn(0x7faf980fcf30 msgr2=0x7faf98103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:13.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 -- 172.21.15.136:0/3398287664 shutdown_connections 2024-04-25T13:14:13.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:14:13.617+0000 7faf9f677700 1 -- 172.21.15.136:0/3398287664 wait complete. 2024-04-25T13:14:14.266 DEBUG:teuthology.orchestra.run.smithi136:osd.5> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T13:14:14.268 INFO:tasks.cephadm:Deploying osd.6 on smithi031 with /dev/vg_nvme/lv_2... 2024-04-25T13:14:14.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:14:14.530 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:14:14.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:14.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:14.828 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:14.829 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:14 smithi031 ceph-mon[29457]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:14.829 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:14 smithi031 ceph-mon[29457]: from='osd.5 [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:14:14.829 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:14 smithi031 ceph-mon[29457]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:14:14.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:14.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:14.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:14.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:14 smithi066 ceph-mon[33137]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:14.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:14 smithi066 ceph-mon[33137]: from='osd.5 [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:14:14.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:14 smithi066 ceph-mon[33137]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:14:14.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:14.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:14.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:14.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:14 smithi136 ceph-mon[33038]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:14.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:14 smithi136 ceph-mon[33038]: from='osd.5 [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:14:14.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:14 smithi136 ceph-mon[33038]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:14:15.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:15 smithi031 ceph-mon[29457]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:14:15.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:15 smithi031 ceph-mon[29457]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T13:14:15.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:15 smithi031 ceph-mon[29457]: from='osd.5 [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:14:15.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:15 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:15.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:15 smithi031 ceph-mon[29457]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:14:15.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:15 smithi066 ceph-mon[33137]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:14:15.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:15 smithi066 ceph-mon[33137]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T13:14:15.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:15 smithi066 ceph-mon[33137]: from='osd.5 [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:14:15.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:15 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:15.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:15 smithi066 ceph-mon[33137]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:14:15.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:15 smithi136 ceph-mon[33038]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:14:15.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:15 smithi136 ceph-mon[33038]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T13:14:15.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:15 smithi136 ceph-mon[33038]: from='osd.5 [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:14:15.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:15 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:15.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:15 smithi136 ceph-mon[33038]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:14:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:14:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:14:16.457 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T13:14:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T13:14:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111907 s, 937 MB/s 2024-04-25T13:14:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T13:14:16.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:16 smithi136 ceph-mon[33038]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:14:16.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:16 smithi136 ceph-mon[33038]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T13:14:16.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:16 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:16.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:16 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:16.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:16 smithi136 ceph-mon[33038]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:16.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:16 smithi136 ceph-mon[33038]: from='osd.5 ' entity='osd.5' 2024-04-25T13:14:16.719 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:14:16 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:14:16.369+0000 7f816f1e0700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T13:14:16.719 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:14:16 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:14:16.451+0000 7f8166fcb700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:14:16.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:16 smithi031 ceph-mon[29457]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:14:16.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:16 smithi031 ceph-mon[29457]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T13:14:16.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:16 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:16.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:16 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:16.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:16 smithi031 ceph-mon[29457]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:16.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:16 smithi031 ceph-mon[29457]: from='osd.5 ' entity='osd.5' 2024-04-25T13:14:16.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:16 smithi066 ceph-mon[33137]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:14:16.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:16 smithi066 ceph-mon[33137]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T13:14:16.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:16 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:16.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:16 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:16.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:16 smithi066 ceph-mon[33137]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:14:16.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:16 smithi066 ceph-mon[33137]: from='osd.5 ' entity='osd.5' 2024-04-25T13:14:17.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2024-04-25T13:14:17.407 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:14:17.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:14:17.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:14:17.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:17.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: osd.5 [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911] boot 2024-04-25T13:14:17.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:14:17.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:17.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi136 2024-04-25T13:14:17.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:17.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:17.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:17.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:17.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi136 to 8544M 2024-04-25T13:14:17.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:17.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:17.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:17.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: osd.5 [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911] boot 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi136 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi136 to 8544M 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:17.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:17.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:17.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: osd.5 [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911] boot 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi136 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:17.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi136 to 8544M 2024-04-25T13:14:17.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:17.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:17.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:17.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:18.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:18 smithi031 ceph-mon[29457]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T13:14:18.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:18 smithi031 ceph-mon[29457]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:18.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:18 smithi066 ceph-mon[33137]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T13:14:18.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:18 smithi066 ceph-mon[33137]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.944+0000 7f00c282b700 1 -- 172.21.15.31:0/3108416575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc102bc0 msgr2=0x7f00bc103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.944+0000 7f00c282b700 1 --2- 172.21.15.31:0/3108416575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc102bc0 0x7f00bc103020 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f00b00099c0 tx=0x7f00b0009cd0 comp rx=0 tx=0).stop 2024-04-25T13:14:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 -- 172.21.15.31:0/3108416575 shutdown_connections 2024-04-25T13:14:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 --2- 172.21.15.31:0/3108416575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00bc103560 0x7f00bc109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 --2- 172.21.15.31:0/3108416575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00bc1019c0 0x7f00bc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 --2- 172.21.15.31:0/3108416575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc102bc0 0x7f00bc103020 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 -- 172.21.15.31:0/3108416575 >> 172.21.15.31:0/3108416575 conn(0x7f00bc0fcf60 msgr2=0x7f00bc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 -- 172.21.15.31:0/3108416575 shutdown_connections 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 -- 172.21.15.31:0/3108416575 wait complete. 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 Processor -- start 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 -- start start 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc1019c0 0x7f00bc105a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00bc102bc0 0x7f00bc104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.945+0000 7f00c282b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00bc103560 0x7f00bc1045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00c282b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10d4b0 con 0x7f00bc1019c0 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00c282b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10d330 con 0x7f00bc102bc0 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00c282b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10d630 con 0x7f00bc103560 2024-04-25T13:14:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc1019c0 0x7f00bc105a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc1019c0 0x7f00bc105a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41284/0 (socket says 172.21.15.31:41284) 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00bbfff700 1 -- 172.21.15.31:0/2867270422 learned_addr learned my addr 172.21.15.31:0/2867270422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00bb7fe700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00bc102bc0 0x7f00bc104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00c0dc8700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00bc103560 0x7f00bc1045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00bbfff700 1 -- 172.21.15.31:0/2867270422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00bc103560 msgr2=0x7f00bc1045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00bbfff700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00bc103560 0x7f00bc1045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00bbfff700 1 -- 172.21.15.31:0/2867270422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00bc102bc0 msgr2=0x7f00bc104070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00bbfff700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00bc102bc0 0x7f00bc104070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00bbfff700 1 -- 172.21.15.31:0/2867270422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc104e60 con 0x7f00bc1019c0 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00bbfff700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc1019c0 0x7f00bc105a20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00c990 tx=0x7f00ac00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00b97fa700 1 -- 172.21.15.31:0/2867270422 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac007b00 con 0x7f00bc1019c0 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00c282b700 1 -- 172.21.15.31:0/2867270422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc1a2d40 con 0x7f00bc1019c0 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00b97fa700 1 -- 172.21.15.31:0/2867270422 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00ac007ca0 con 0x7f00bc1019c0 2024-04-25T13:14:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00b97fa700 1 -- 172.21.15.31:0/2867270422 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac018660 con 0x7f00bc1019c0 2024-04-25T13:14:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.946+0000 7f00c282b700 1 -- 172.21.15.31:0/2867270422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1a31c0 con 0x7f00bc1019c0 2024-04-25T13:14:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.947+0000 7f00c282b700 1 -- 172.21.15.31:0/2867270422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc069730 con 0x7f00bc1019c0 2024-04-25T13:14:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.949+0000 7f00b97fa700 1 -- 172.21.15.31:0/2867270422 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f00ac00f450 con 0x7f00bc1019c0 2024-04-25T13:14:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.950+0000 7f00b97fa700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f00a40a3c40 0x7f00a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.950+0000 7f00bb7fe700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f00a40a3c40 0x7f00a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.950+0000 7f00b97fa700 1 -- 172.21.15.31:0/2867270422 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4562+0+0 (secure 0 0 0) 0x7f00ac0cf570 con 0x7f00bc1019c0 2024-04-25T13:14:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.950+0000 7f00bb7fe700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f00a40a3c40 0x7f00a40a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009990 tx=0x7f00b000b910 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:18.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:18.954+0000 7f00b97fa700 1 -- 172.21.15.31:0/2867270422 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00ac014030 con 0x7f00bc1019c0 2024-04-25T13:14:18.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:18 smithi136 ceph-mon[33038]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T13:14:18.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:18 smithi136 ceph-mon[33038]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:19.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:19.171+0000 7f00c282b700 1 -- 172.21.15.31:0/2867270422 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f00bc06b880 con 0x7f00a40a3c40 2024-04-25T13:14:19.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:19 smithi031 ceph-mon[29457]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T13:14:19.895 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:19 smithi031 ceph-mon[29457]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:19.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:14:19.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:14:19.896 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:19.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:19 smithi066 ceph-mon[33137]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T13:14:19.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:19 smithi066 ceph-mon[33137]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:19.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:14:19.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:14:19.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:19.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:19 smithi136 ceph-mon[33038]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T13:14:19.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:19 smithi136 ceph-mon[33038]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:19.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:14:19.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:14:19.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:20.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:20 smithi031 ceph-mon[29457]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T13:14:20.898 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:20 smithi031 ceph-mon[29457]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:20.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:20 smithi066 ceph-mon[33137]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T13:14:20.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:20 smithi066 ceph-mon[33137]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:20.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:20 smithi136 ceph-mon[33038]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T13:14:20.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:20 smithi136 ceph-mon[33038]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:21.907 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:21 smithi066 ceph-mon[33137]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T13:14:21.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:21 smithi136 ceph-mon[33038]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T13:14:21.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:21 smithi031 ceph-mon[29457]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T13:14:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:22 smithi066 ceph-mon[33137]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-25T13:14:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:22 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/833650296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00c2e910-8c20-44b2-ab6d-2eb18f8d8add"}]: dispatch 2024-04-25T13:14:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:22 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/833650296' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00c2e910-8c20-44b2-ab6d-2eb18f8d8add"}]': finished 2024-04-25T13:14:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:22 smithi066 ceph-mon[33137]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T13:14:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:22 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/927194746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:14:22.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:22 smithi136 ceph-mon[33038]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-25T13:14:22.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:22 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/833650296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00c2e910-8c20-44b2-ab6d-2eb18f8d8add"}]: dispatch 2024-04-25T13:14:22.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:22 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/833650296' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00c2e910-8c20-44b2-ab6d-2eb18f8d8add"}]': finished 2024-04-25T13:14:22.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:22 smithi136 ceph-mon[33038]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T13:14:22.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:22.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:22 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/927194746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:14:22.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:22 smithi031 ceph-mon[29457]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-25T13:14:22.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:22 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/833650296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00c2e910-8c20-44b2-ab6d-2eb18f8d8add"}]: dispatch 2024-04-25T13:14:22.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:22 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/833650296' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00c2e910-8c20-44b2-ab6d-2eb18f8d8add"}]': finished 2024-04-25T13:14:22.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:22 smithi031 ceph-mon[29457]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T13:14:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:22 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/927194746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:14:25.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:24 smithi066 ceph-mon[33137]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-25T13:14:25.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:24 smithi136 ceph-mon[33038]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-25T13:14:25.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:24 smithi031 ceph-mon[29457]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-25T13:14:26.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:26 smithi066 ceph-mon[33137]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T13:14:26.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:26 smithi031 ceph-mon[29457]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T13:14:27.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:26 smithi136 ceph-mon[33038]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T13:14:28.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:28 smithi031 ceph-mon[29457]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:14:29.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:28 smithi066 ceph-mon[33137]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:14:29.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:28 smithi136 ceph-mon[33038]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:14:31.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:30 smithi066 ceph-mon[33137]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T13:14:31.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:30 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:14:31.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:30 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:31.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:30 smithi066 ceph-mon[33137]: Deploying daemon osd.6 on smithi031 2024-04-25T13:14:31.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:30 smithi136 ceph-mon[33038]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T13:14:31.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:30 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:14:31.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:30 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:31.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:30 smithi136 ceph-mon[33038]: Deploying daemon osd.6 on smithi031 2024-04-25T13:14:31.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:30 smithi031 ceph-mon[29457]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T13:14:31.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:30 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:14:31.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:30 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:31.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:30 smithi031 ceph-mon[29457]: Deploying daemon osd.6 on smithi031 2024-04-25T13:14:32.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:32 smithi031 ceph-mon[29457]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:33.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:32 smithi066 ceph-mon[33137]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:33.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:32 smithi136 ceph-mon[33038]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:35.151 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:34 smithi031 ceph-mon[29457]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:35.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:34 smithi066 ceph-mon[33137]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:35.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:34 smithi136 ceph-mon[33038]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:36.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:36 smithi031 ceph-mon[29457]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:37.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:36 smithi066 ceph-mon[33137]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:37.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:36 smithi136 ceph-mon[33038]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:39.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:38 smithi031 ceph-mon[29457]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:39.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:38 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:39.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:38 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:39.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:38 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:39.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:38 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:39.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:38 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:38 smithi066 ceph-mon[33137]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:38 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:38 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:38 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:38 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:38 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:38 smithi136 ceph-mon[33038]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:38 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:38 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:38 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:38 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:38 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:39.887 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 6 on host 'smithi031' 2024-04-25T13:14:39.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.886+0000 7f00b97fa700 1 -- 172.21.15.31:0/2867270422 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f00bc06b880 con 0x7f00a40a3c40 2024-04-25T13:14:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 -- 172.21.15.31:0/2867270422 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f00a40a3c40 msgr2=0x7f00a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f00a40a3c40 0x7f00a40a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009990 tx=0x7f00b000b910 comp rx=0 tx=0).stop 2024-04-25T13:14:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 -- 172.21.15.31:0/2867270422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc1019c0 msgr2=0x7f00bc105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc1019c0 0x7f00bc105a20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00c990 tx=0x7f00ac00cca0 comp rx=0 tx=0).stop 2024-04-25T13:14:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 -- 172.21.15.31:0/2867270422 shutdown_connections 2024-04-25T13:14:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f00a40a3c40 0x7f00a40a6100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00bc103560 0x7f00bc1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:39.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc1019c0 0x7f00bc105a20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:39.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 --2- 172.21.15.31:0/2867270422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00bc102bc0 0x7f00bc104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:39.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 -- 172.21.15.31:0/2867270422 >> 172.21.15.31:0/2867270422 conn(0x7f00bc0fcf60 msgr2=0x7f00bc06b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:39.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 -- 172.21.15.31:0/2867270422 shutdown_connections 2024-04-25T13:14:39.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:14:39.888+0000 7f00c282b700 1 -- 172.21.15.31:0/2867270422 wait complete. 2024-04-25T13:14:40.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:39 smithi066 ceph-mon[33137]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:14:40.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:39 smithi136 ceph-mon[33038]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:14:40.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:39 smithi031 ceph-mon[29457]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:14:40.656 DEBUG:teuthology.orchestra.run.smithi031:osd.6> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T13:14:40.658 INFO:tasks.cephadm:Deploying osd.7 on smithi066 with /dev/vg_nvme/lv_2... 2024-04-25T13:14:40.659 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:14:40.916 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:14:40.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:40 smithi031 ceph-mon[29457]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:40.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:40 smithi031 ceph-mon[29457]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:14:40.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:40 smithi031 ceph-mon[29457]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T13:14:40.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:40 smithi031 ceph-mon[29457]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:14:40.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:40.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:40.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:40.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:41.039 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:40 smithi066 ceph-mon[33137]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:41.039 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:40 smithi066 ceph-mon[33137]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:14:41.040 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:40 smithi066 ceph-mon[33137]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T13:14:41.040 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:40 smithi066 ceph-mon[33137]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:14:41.040 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:41.040 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:41.040 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:41.040 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:41.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:40 smithi136 ceph-mon[33038]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:41.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:40 smithi136 ceph-mon[33038]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:14:41.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:40 smithi136 ceph-mon[33038]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T13:14:41.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:40 smithi136 ceph-mon[33038]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:14:41.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:41.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:41.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:41.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:14:42.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:41 smithi031 ceph-mon[29457]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:14:42.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:41 smithi031 ceph-mon[29457]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T13:14:42.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:41 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:42.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:41 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:42.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:41 smithi031 ceph-mon[29457]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' 2024-04-25T13:14:42.087 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:14:41 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:14:41.750+0000 7fa9e1c7c700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T13:14:42.088 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:14:41 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:14:41.842+0000 7fa9d9a67700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:14:42.088 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:41 smithi066 ceph-mon[33137]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:14:42.088 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:41 smithi066 ceph-mon[33137]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T13:14:42.088 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:41 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:42.088 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:41 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:42.088 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:41 smithi066 ceph-mon[33137]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' 2024-04-25T13:14:42.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:41 smithi136 ceph-mon[33038]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:14:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:41 smithi136 ceph-mon[33038]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T13:14:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:41 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:41 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:41 smithi136 ceph-mon[33038]: from='osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141]' entity='osd.6' 2024-04-25T13:14:42.941 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:42 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:14:42.941 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:42 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:14:42.941 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:42 smithi066 ceph-mon[33137]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:42.941 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:42.941 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:42 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:42.941 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:14:42.941 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:14:42.941 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-04-25T13:14:42.941 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-04-25T13:14:42.942 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112063 s, 936 MB/s 2024-04-25T13:14:42.942 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-04-25T13:14:43.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:42 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:14:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:42 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:14:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:42 smithi136 ceph-mon[33038]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:42 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:42 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:14:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:42 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:14:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:42 smithi031 ceph-mon[29457]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:42 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:43.725 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_2 2024-04-25T13:14:43.982 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:14:44.020 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141] boot 2024-04-25T13:14:44.020 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T13:14:44.020 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:44.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:44.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:44.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:44.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:44.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:44.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:44.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:44.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:44.021 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:43 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:44.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141] boot 2024-04-25T13:14:44.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T13:14:44.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:44.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:44.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:44.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:44.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:44.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:44.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:44.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:44.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:44.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:43 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:44.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: osd.6 [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141] boot 2024-04-25T13:14:44.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T13:14:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:14:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:14:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:14:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:43 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:14:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:44 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi031 2024-04-25T13:14:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:44 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi031 to 5697M 2024-04-25T13:14:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:44 smithi066 ceph-mon[33137]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:44 smithi066 ceph-mon[33137]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T13:14:45.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:44 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi031 2024-04-25T13:14:45.313 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:44 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi031 to 5697M 2024-04-25T13:14:45.313 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:44 smithi136 ceph-mon[33038]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:45.313 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:44 smithi136 ceph-mon[33038]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T13:14:45.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:44 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi031 2024-04-25T13:14:45.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:44 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi031 to 5697M 2024-04-25T13:14:45.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:44 smithi031 ceph-mon[29457]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:14:45.319 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:44 smithi031 ceph-mon[29457]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T13:14:45.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.751+0000 7fd26f742700 1 -- 172.21.15.66:0/2861070206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd268102ac0 msgr2=0x7fd268102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:45.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.751+0000 7fd26f742700 1 --2- 172.21.15.66:0/2861070206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd268102ac0 0x7fd268102ee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd2640099c0 tx=0x7fd264009cd0 comp rx=0 tx=0).stop 2024-04-25T13:14:45.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.751+0000 7fd26f742700 1 -- 172.21.15.66:0/2861070206 shutdown_connections 2024-04-25T13:14:45.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.751+0000 7fd26f742700 1 --2- 172.21.15.66:0/2861070206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268104660 0x7fd26810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:45.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.751+0000 7fd26f742700 1 --2- 172.21.15.66:0/2861070206 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd268102ac0 0x7fd268102ee0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:45.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.751+0000 7fd26f742700 1 --2- 172.21.15.66:0/2861070206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd268103cc0 0x7fd268104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:45.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.751+0000 7fd26f742700 1 -- 172.21.15.66:0/2861070206 >> 172.21.15.66:0/2861070206 conn(0x7fd2680fe060 msgr2=0x7fd2681004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:14:45.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.751+0000 7fd26f742700 1 -- 172.21.15.66:0/2861070206 shutdown_connections 2024-04-25T13:14:45.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.751+0000 7fd26f742700 1 -- 172.21.15.66:0/2861070206 wait complete. 2024-04-25T13:14:45.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.752+0000 7fd26f742700 1 Processor -- start 2024-04-25T13:14:45.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.752+0000 7fd26f742700 1 -- start start 2024-04-25T13:14:45.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.752+0000 7fd26f742700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268102ac0 0x7fd268198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:45.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.752+0000 7fd26f742700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd268103cc0 0x7fd268199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:45.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.752+0000 7fd26f742700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd268104660 0x7fd2681a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:45.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.752+0000 7fd26f742700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810e550 con 0x7fd268104660 2024-04-25T13:14:45.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.752+0000 7fd26f742700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810e3d0 con 0x7fd268103cc0 2024-04-25T13:14:45.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.752+0000 7fd26f742700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810e6d0 con 0x7fd268102ac0 2024-04-25T13:14:45.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.752+0000 7fd26ccdd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd268103cc0 0x7fd268199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:45.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.752+0000 7fd26ccdd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd268103cc0 0x7fd268199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41666/0 (socket says 172.21.15.66:41666) 2024-04-25T13:14:45.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd26ccdd700 1 -- 172.21.15.66:0/3302835938 learned_addr learned my addr 172.21.15.66:0/3302835938 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-25T13:14:45.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd26d4de700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268102ac0 0x7fd268198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:45.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd26dcdf700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd268104660 0x7fd2681a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:45.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd26ccdd700 1 -- 172.21.15.66:0/3302835938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268102ac0 msgr2=0x7fd268198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:45.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd26ccdd700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268102ac0 0x7fd268198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:45.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd26ccdd700 1 -- 172.21.15.66:0/3302835938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd268104660 msgr2=0x7fd2681a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:14:45.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd26ccdd700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd268104660 0x7fd2681a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:14:45.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd26ccdd700 1 -- 172.21.15.66:0/3302835938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2681a0840 con 0x7fd268103cc0 2024-04-25T13:14:45.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd26ccdd700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd268103cc0 0x7fd268199120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c920 tx=0x7fd25800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:45.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd25e7fc700 1 -- 172.21.15.66:0/3302835938 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258007b00 con 0x7fd268103cc0 2024-04-25T13:14:45.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd25e7fc700 1 -- 172.21.15.66:0/3302835938 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd258004ce0 con 0x7fd268103cc0 2024-04-25T13:14:45.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd26f742700 1 -- 172.21.15.66:0/3302835938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2681a0b30 con 0x7fd268103cc0 2024-04-25T13:14:45.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd25e7fc700 1 -- 172.21.15.66:0/3302835938 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258005580 con 0x7fd268103cc0 2024-04-25T13:14:45.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.753+0000 7fd26f742700 1 -- 172.21.15.66:0/3302835938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2681a10c0 con 0x7fd268103cc0 2024-04-25T13:14:45.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.754+0000 7fd26f742700 1 -- 172.21.15.66:0/3302835938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd268069730 con 0x7fd268103cc0 2024-04-25T13:14:45.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.755+0000 7fd25e7fc700 1 -- 172.21.15.66:0/3302835938 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd258004750 con 0x7fd268103cc0 2024-04-25T13:14:45.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.756+0000 7fd25e7fc700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd2540a3cc0 0x7fd2540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:14:45.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.756+0000 7fd25e7fc700 1 -- 172.21.15.66:0/3302835938 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7fd2580cf170 con 0x7fd268103cc0 2024-04-25T13:14:45.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.756+0000 7fd26d4de700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd2540a3cc0 0x7fd2540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:14:45.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.757+0000 7fd26d4de700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd2540a3cc0 0x7fd2540a6180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2640038c0 tx=0x7fd26400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:14:45.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.761+0000 7fd25e7fc700 1 -- 172.21.15.66:0/3302835938 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd258014030 con 0x7fd268103cc0 2024-04-25T13:14:45.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:14:45.978+0000 7fd26f742700 1 -- 172.21.15.66:0/3302835938 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd268062e80 con 0x7fd2540a3cc0 2024-04-25T13:14:46.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:46 smithi066 ceph-mon[33137]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:46.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:46 smithi066 ceph-mon[33137]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:46.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:14:46.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:14:46.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:47.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:46 smithi136 ceph-mon[33038]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:46 smithi136 ceph-mon[33038]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:14:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:14:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:47.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:46 smithi031 ceph-mon[29457]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:47.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:46 smithi031 ceph-mon[29457]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:14:47.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:14:47.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:14:47.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:48 smithi066 ceph-mon[33137]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:48 smithi066 ceph-mon[33137]: from='client.? 172.21.15.66:0/3839061484' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50858634-4ead-4b51-9ed1-e5dfe3fd2ceb"}]: dispatch 2024-04-25T13:14:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:48 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50858634-4ead-4b51-9ed1-e5dfe3fd2ceb"}]: dispatch 2024-04-25T13:14:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:48 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50858634-4ead-4b51-9ed1-e5dfe3fd2ceb"}]': finished 2024-04-25T13:14:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:48 smithi066 ceph-mon[33137]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:14:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:48 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:14:49.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:48 smithi136 ceph-mon[33038]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:49.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:48 smithi136 ceph-mon[33038]: from='client.? 172.21.15.66:0/3839061484' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50858634-4ead-4b51-9ed1-e5dfe3fd2ceb"}]: dispatch 2024-04-25T13:14:49.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:48 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50858634-4ead-4b51-9ed1-e5dfe3fd2ceb"}]: dispatch 2024-04-25T13:14:49.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:48 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50858634-4ead-4b51-9ed1-e5dfe3fd2ceb"}]': finished 2024-04-25T13:14:49.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:48 smithi136 ceph-mon[33038]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:14:49.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:48 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:14:49.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:48 smithi031 ceph-mon[29457]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:49.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:48 smithi031 ceph-mon[29457]: from='client.? 172.21.15.66:0/3839061484' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50858634-4ead-4b51-9ed1-e5dfe3fd2ceb"}]: dispatch 2024-04-25T13:14:49.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:48 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50858634-4ead-4b51-9ed1-e5dfe3fd2ceb"}]: dispatch 2024-04-25T13:14:49.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:48 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50858634-4ead-4b51-9ed1-e5dfe3fd2ceb"}]': finished 2024-04-25T13:14:49.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:48 smithi031 ceph-mon[29457]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:14:49.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:48 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:14:50.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:49 smithi066 ceph-mon[33137]: from='client.? 172.21.15.66:0/3405290196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:14:50.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:49 smithi136 ceph-mon[33038]: from='client.? 172.21.15.66:0/3405290196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:14:50.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:49 smithi031 ceph-mon[29457]: from='client.? 172.21.15.66:0/3405290196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:14:51.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:50 smithi066 ceph-mon[33137]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:51.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:50 smithi136 ceph-mon[33038]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:51.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:50 smithi031 ceph-mon[29457]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:53.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:52 smithi066 ceph-mon[33137]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:53.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:52 smithi136 ceph-mon[33038]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:53.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:52 smithi031 ceph-mon[29457]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:55.079 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:54 smithi066 ceph-mon[33137]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:55.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:54 smithi136 ceph-mon[33038]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:55.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:54 smithi031 ceph-mon[29457]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:57.144 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:56 smithi066 ceph-mon[33137]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:57.144 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:56 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:14:57.144 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:56 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:57.145 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:56 smithi066 ceph-mon[33137]: Deploying daemon osd.7 on smithi066 2024-04-25T13:14:57.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:56 smithi136 ceph-mon[33038]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:56 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:14:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:56 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:56 smithi136 ceph-mon[33038]: Deploying daemon osd.7 on smithi066 2024-04-25T13:14:57.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:56 smithi031 ceph-mon[29457]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:57.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:56 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:14:57.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:56 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:14:57.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:56 smithi031 ceph-mon[29457]: Deploying daemon osd.7 on smithi066 2024-04-25T13:14:59.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:14:58 smithi066 ceph-mon[33137]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:59.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:14:58 smithi136 ceph-mon[33038]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:14:59.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:14:58 smithi031 ceph-mon[29457]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:01.014 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:00 smithi066 ceph-mon[33137]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:01.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:00 smithi136 ceph-mon[33038]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:01.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:00 smithi031 ceph-mon[29457]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:02 smithi066 ceph-mon[33137]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:03.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:02 smithi136 ceph-mon[33038]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:03.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:02 smithi031 ceph-mon[29457]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:03.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:03.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:03.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:05.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:04 smithi066 ceph-mon[33137]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:05.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:05.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:05.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:05.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:05.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:04 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:04 smithi136 ceph-mon[33038]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:04 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:05.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:04 smithi031 ceph-mon[29457]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:05.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:05.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:05.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:05.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:05.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:04 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:05.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.742+0000 7fd25e7fc700 1 -- 172.21.15.66:0/3302835938 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd268062e80 con 0x7fd2540a3cc0 2024-04-25T13:15:05.743 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 7 on host 'smithi066' 2024-04-25T13:15:05.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 -- 172.21.15.66:0/3302835938 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd2540a3cc0 msgr2=0x7fd2540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:05.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd2540a3cc0 0x7fd2540a6180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2640038c0 tx=0x7fd26400b540 comp rx=0 tx=0).stop 2024-04-25T13:15:05.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 -- 172.21.15.66:0/3302835938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd268103cc0 msgr2=0x7fd268199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:05.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd268103cc0 0x7fd268199120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c920 tx=0x7fd25800cc30 comp rx=0 tx=0).stop 2024-04-25T13:15:05.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 -- 172.21.15.66:0/3302835938 shutdown_connections 2024-04-25T13:15:05.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd2540a3cc0 0x7fd2540a6180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:05.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268102ac0 0x7fd268198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:05.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd268104660 0x7fd2681a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:05.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 --2- 172.21.15.66:0/3302835938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd268103cc0 0x7fd268199120 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:05.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 -- 172.21.15.66:0/3302835938 >> 172.21.15.66:0/3302835938 conn(0x7fd2680fe060 msgr2=0x7fd268107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:05.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 -- 172.21.15.66:0/3302835938 shutdown_connections 2024-04-25T13:15:05.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:15:05.745+0000 7fd26f742700 1 -- 172.21.15.66:0/3302835938 wait complete. 2024-04-25T13:15:06.362 DEBUG:teuthology.orchestra.run.smithi066:osd.7> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T13:15:06.364 INFO:tasks.cephadm:Deploying osd.8 on smithi136 with /dev/vg_nvme/lv_2... 2024-04-25T13:15:06.364 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:15:06.612 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:15:06.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:06 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:06.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:06 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:06.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:06 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:06.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:06 smithi066 ceph-mon[33137]: from='osd.7 [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:15:06.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:06 smithi066 ceph-mon[33137]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:15:06.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:06 smithi066 ceph-mon[33137]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:06.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:06 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:06.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:06 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:06.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:06 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:06.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:06 smithi136 ceph-mon[33038]: from='osd.7 [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:15:06.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:06 smithi136 ceph-mon[33038]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:15:06.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:06 smithi136 ceph-mon[33038]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:06.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:06 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:06.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:06 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:06.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:06 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:06.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:06 smithi031 ceph-mon[29457]: from='osd.7 [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:15:06.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:06 smithi031 ceph-mon[29457]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:15:06.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:06 smithi031 ceph-mon[29457]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:07.826 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:07 smithi066 ceph-mon[33137]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:15:07.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:07 smithi066 ceph-mon[33137]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:15:07.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:07 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:07.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:07 smithi066 ceph-mon[33137]: from='osd.7 [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:15:07.827 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:07 smithi066 ceph-mon[33137]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:15:07.827 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:15:07 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:15:07.710+0000 7f79596ed700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T13:15:07.827 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:15:07 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:15:07.722+0000 7f79554e0700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:15:07.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:07 smithi136 ceph-mon[33038]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:15:07.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:07 smithi136 ceph-mon[33038]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:15:07.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:07 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:07.928 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:07 smithi136 ceph-mon[33038]: from='osd.7 [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:15:07.928 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:07 smithi136 ceph-mon[33038]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:15:07.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:07 smithi031 ceph-mon[29457]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:15:07.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:07 smithi031 ceph-mon[29457]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:15:07.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:07 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:07.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:07 smithi031 ceph-mon[29457]: from='osd.7 [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:15:07.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:07 smithi031 ceph-mon[29457]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:15:08.534 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:15:08.535 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:15:08.535 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-04-25T13:15:08.535 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-04-25T13:15:08.535 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00863267 s, 1.2 GB/s 2024-04-25T13:15:08.535 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-04-25T13:15:08.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:08 smithi136 ceph-mon[33038]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:15:08.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:08 smithi136 ceph-mon[33038]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T13:15:08.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:08 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:08.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:08 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:08.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:08 smithi136 ceph-mon[33038]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:08 smithi031 ceph-mon[29457]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:15:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:08 smithi031 ceph-mon[29457]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T13:15:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:08 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:08 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:08 smithi031 ceph-mon[29457]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:09.023 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:08 smithi066 ceph-mon[33137]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:15:09.023 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:08 smithi066 ceph-mon[33137]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T13:15:09.023 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:08 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:09.023 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:08 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:09.024 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:08 smithi066 ceph-mon[33137]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:15:09.132 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_2 2024-04-25T13:15:09.371 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:15:09.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:15:09.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:15:09.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: osd.7 [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945] boot 2024-04-25T13:15:09.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T13:15:09.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:09.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi066 2024-04-25T13:15:09.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:09.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:09.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:09.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:09.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:09.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi066 to 5696M 2024-04-25T13:15:09.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:09.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:09.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:09.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:09.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:09 smithi136 ceph-mon[33038]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:15:09.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: osd.7 [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945] boot 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi066 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi066 to 5696M 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:09.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:09.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:09.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:09 smithi031 ceph-mon[29457]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:15:10.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:15:10.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:15:10.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: osd.7 [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945] boot 2024-04-25T13:15:10.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T13:15:10.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:15:10.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi066 2024-04-25T13:15:10.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:10.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:10.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:10.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:10.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:10.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi066 to 5696M 2024-04-25T13:15:10.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:10.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:10.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:10.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:10.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:09 smithi066 ceph-mon[33137]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:15:10.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.941+0000 7efc5b87b700 1 -- 172.21.15.136:0/709951313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc54104640 msgr2=0x7efc5410aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:10.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.941+0000 7efc5b87b700 1 --2- 172.21.15.136:0/709951313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc54104640 0x7efc5410aef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00b0a0 tx=0x7efc4c00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:15:10.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.941+0000 7efc5b87b700 1 -- 172.21.15.136:0/709951313 shutdown_connections 2024-04-25T13:15:10.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.941+0000 7efc5b87b700 1 --2- 172.21.15.136:0/709951313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc54104640 0x7efc5410aef0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:10.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.941+0000 7efc5b87b700 1 --2- 172.21.15.136:0/709951313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc54102aa0 0x7efc54102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:10.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.941+0000 7efc5b87b700 1 --2- 172.21.15.136:0/709951313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc54103ca0 0x7efc54104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:10.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.941+0000 7efc5b87b700 1 -- 172.21.15.136:0/709951313 >> 172.21.15.136:0/709951313 conn(0x7efc540fe040 msgr2=0x7efc54100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.942+0000 7efc5b87b700 1 -- 172.21.15.136:0/709951313 shutdown_connections 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.942+0000 7efc5b87b700 1 -- 172.21.15.136:0/709951313 wait complete. 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.942+0000 7efc5b87b700 1 Processor -- start 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.942+0000 7efc5b87b700 1 -- start start 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.942+0000 7efc5b87b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc54102aa0 0x7efc540767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.942+0000 7efc5b87b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc54103ca0 0x7efc54074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.942+0000 7efc5b87b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc54104640 0x7efc54075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.942+0000 7efc5b87b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efc5410e630 con 0x7efc54104640 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.942+0000 7efc5b87b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efc5410e4b0 con 0x7efc54102aa0 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.942+0000 7efc5b87b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc5410e7b0 con 0x7efc54103ca0 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc58e16700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc54103ca0 0x7efc54074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc58e16700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc54103ca0 0x7efc54074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:50752/0 (socket says 172.21.15.136:50752) 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc58e16700 1 -- 172.21.15.136:0/195294829 learned_addr learned my addr 172.21.15.136:0/195294829 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T13:15:10.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc59e18700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc54104640 0x7efc54075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc59617700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc54102aa0 0x7efc540767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc58e16700 1 -- 172.21.15.136:0/195294829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc54102aa0 msgr2=0x7efc540767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc58e16700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc54102aa0 0x7efc540767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc58e16700 1 -- 172.21.15.136:0/195294829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc54104640 msgr2=0x7efc54075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc58e16700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc54104640 0x7efc54075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc58e16700 1 -- 172.21.15.136:0/195294829 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc54075c20 con 0x7efc54103ca0 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc59e18700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc54104640 0x7efc54075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc58e16700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc54103ca0 0x7efc54074e30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b530 tx=0x7efc4400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc4a7fc700 1 -- 172.21.15.136:0/195294829 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44015020 con 0x7efc54103ca0 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc4a7fc700 1 -- 172.21.15.136:0/195294829 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc44011ce0 con 0x7efc54103ca0 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc5b87b700 1 -- 172.21.15.136:0/195294829 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc541a2ec0 con 0x7efc54103ca0 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc4a7fc700 1 -- 172.21.15.136:0/195294829 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44010d20 con 0x7efc54103ca0 2024-04-25T13:15:10.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.943+0000 7efc5b87b700 1 -- 172.21.15.136:0/195294829 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc541a33a0 con 0x7efc54103ca0 2024-04-25T13:15:10.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.944+0000 7efc5b87b700 1 -- 172.21.15.136:0/195294829 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc54069730 con 0x7efc54103ca0 2024-04-25T13:15:10.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.946+0000 7efc4a7fc700 1 -- 172.21.15.136:0/195294829 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efc44018020 con 0x7efc54103ca0 2024-04-25T13:15:10.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.946+0000 7efc4a7fc700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efc400a3c40 0x7efc400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:10.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.947+0000 7efc4a7fc700 1 -- 172.21.15.136:0/195294829 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5426+0+0 (secure 0 0 0) 0x7efc440cf5f0 con 0x7efc54103ca0 2024-04-25T13:15:10.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.947+0000 7efc59617700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efc400a3c40 0x7efc400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:10.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.947+0000 7efc59617700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efc400a3c40 0x7efc400a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc50006fd0 tx=0x7efc50008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:10.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:10.950+0000 7efc4a7fc700 1 -- 172.21.15.136:0/195294829 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc4401f020 con 0x7efc54103ca0 2024-04-25T13:15:10.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:10 smithi136 ceph-mon[33038]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:10.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:10 smithi031 ceph-mon[29457]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:10 smithi066 ceph-mon[33137]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:11.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:11.158+0000 7efc5b87b700 1 -- 172.21.15.136:0/195294829 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7efc54062e80 con 0x7efc400a3c40 2024-04-25T13:15:11.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:11 smithi031 ceph-mon[29457]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:15:11.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:11 smithi031 ceph-mon[29457]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:15:11.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:15:11.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:15:11.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:11.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:11 smithi136 ceph-mon[33038]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:15:11.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:11 smithi136 ceph-mon[33038]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:15:11.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:11 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:15:11.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:11 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:15:11.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:11 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:12.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:11 smithi066 ceph-mon[33137]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:15:12.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:11 smithi066 ceph-mon[33137]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:15:12.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:11 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:15:12.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:11 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:15:12.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:11 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:12.853 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:12 smithi136 ceph-mon[33038]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:15:12.853 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:12 smithi136 ceph-mon[33038]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:13.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:12 smithi066 ceph-mon[33137]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:15:13.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:12 smithi066 ceph-mon[33137]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:13.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:12 smithi031 ceph-mon[29457]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:15:13.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:12 smithi031 ceph-mon[29457]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:14.059 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:13 smithi066 ceph-mon[33137]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T13:15:14.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:13 smithi136 ceph-mon[33038]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T13:15:14.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:13 smithi031 ceph-mon[29457]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T13:15:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:14 smithi066 ceph-mon[33137]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:14 smithi066 ceph-mon[33137]: from='client.? 172.21.15.136:0/125053505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3649fef3-e745-4d64-9601-f09db9203458"}]: dispatch 2024-04-25T13:15:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:14 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3649fef3-e745-4d64-9601-f09db9203458"}]: dispatch 2024-04-25T13:15:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:14 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3649fef3-e745-4d64-9601-f09db9203458"}]': finished 2024-04-25T13:15:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:14 smithi066 ceph-mon[33137]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T13:15:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:14 smithi066 ceph-mon[33137]: from='client.? 172.21.15.136:0/2526039928' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:15:15.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:14 smithi136 ceph-mon[33038]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:14 smithi136 ceph-mon[33038]: from='client.? 172.21.15.136:0/125053505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3649fef3-e745-4d64-9601-f09db9203458"}]: dispatch 2024-04-25T13:15:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:14 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3649fef3-e745-4d64-9601-f09db9203458"}]: dispatch 2024-04-25T13:15:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:14 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3649fef3-e745-4d64-9601-f09db9203458"}]': finished 2024-04-25T13:15:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:14 smithi136 ceph-mon[33038]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T13:15:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:14 smithi136 ceph-mon[33038]: from='client.? 172.21.15.136:0/2526039928' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:15:15.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:14 smithi031 ceph-mon[29457]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:15.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:14 smithi031 ceph-mon[29457]: from='client.? 172.21.15.136:0/125053505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3649fef3-e745-4d64-9601-f09db9203458"}]: dispatch 2024-04-25T13:15:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:14 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3649fef3-e745-4d64-9601-f09db9203458"}]: dispatch 2024-04-25T13:15:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:14 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3649fef3-e745-4d64-9601-f09db9203458"}]': finished 2024-04-25T13:15:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:14 smithi031 ceph-mon[29457]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T13:15:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:14 smithi031 ceph-mon[29457]: from='client.? 172.21.15.136:0/2526039928' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:15:17.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:16 smithi066 ceph-mon[33137]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:17.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:16 smithi136 ceph-mon[33038]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:17.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:16 smithi031 ceph-mon[29457]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:18.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:18 smithi136 ceph-mon[33038]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:19.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:18 smithi066 ceph-mon[33137]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:19.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:18 smithi031 ceph-mon[29457]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:21.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:20 smithi066 ceph-mon[33137]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:21.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:20 smithi136 ceph-mon[33038]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:21.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:20 smithi031 ceph-mon[29457]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:21.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:21 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:15:21.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:21 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:22.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:21 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:15:22.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:21 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:22.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:21 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:15:22.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:21 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:23.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:22 smithi066 ceph-mon[33137]: Deploying daemon osd.8 on smithi136 2024-04-25T13:15:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:22 smithi066 ceph-mon[33137]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:23.194 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:22 smithi136 ceph-mon[33038]: Deploying daemon osd.8 on smithi136 2024-04-25T13:15:23.194 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:22 smithi136 ceph-mon[33038]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:23.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:22 smithi031 ceph-mon[29457]: Deploying daemon osd.8 on smithi136 2024-04-25T13:15:23.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:22 smithi031 ceph-mon[29457]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:25.094 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:24 smithi136 ceph-mon[33038]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:25.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:24 smithi066 ceph-mon[33137]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:25.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:24 smithi031 ceph-mon[29457]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:26.962 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:26 smithi136 ceph-mon[33038]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:27.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:26 smithi066 ceph-mon[33137]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:27.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:26 smithi031 ceph-mon[29457]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:28.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:28 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:28.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:28 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:28.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:28 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:28.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:28 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:28.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:28 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:28.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:28 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:28.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:28 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:28.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:28 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:28.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:28 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:29.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:29 smithi136 ceph-mon[33038]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:29.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:29 smithi031 ceph-mon[29457]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:29.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:29 smithi066 ceph-mon[33137]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:30.680 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 8 on host 'smithi136' 2024-04-25T13:15:30.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.679+0000 7efc4a7fc700 1 -- 172.21.15.136:0/195294829 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efc54062e80 con 0x7efc400a3c40 2024-04-25T13:15:30.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 -- 172.21.15.136:0/195294829 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efc400a3c40 msgr2=0x7efc400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efc400a3c40 0x7efc400a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc50006fd0 tx=0x7efc50008040 comp rx=0 tx=0).stop 2024-04-25T13:15:30.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 -- 172.21.15.136:0/195294829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc54103ca0 msgr2=0x7efc54074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:30.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc54103ca0 0x7efc54074e30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efc4400b530 tx=0x7efc4400b8f0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 -- 172.21.15.136:0/195294829 shutdown_connections 2024-04-25T13:15:30.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc54104640 0x7efc54075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc54102aa0 0x7efc540767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efc400a3c40 0x7efc400a6100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 --2- 172.21.15.136:0/195294829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc54103ca0 0x7efc54074e30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:30.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 -- 172.21.15.136:0/195294829 >> 172.21.15.136:0/195294829 conn(0x7efc540fe040 msgr2=0x7efc54107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:30.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 -- 172.21.15.136:0/195294829 shutdown_connections 2024-04-25T13:15:30.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:15:30.682+0000 7efc5b87b700 1 -- 172.21.15.136:0/195294829 wait complete. 2024-04-25T13:15:31.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:31 smithi136 ceph-mon[33038]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:31.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:31.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:31.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:31 smithi136 ceph-mon[33038]: from='osd.8 [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:15:31.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:31 smithi136 ceph-mon[33038]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:15:31.354 DEBUG:teuthology.orchestra.run.smithi136:osd.8> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T13:15:31.357 INFO:tasks.cephadm:Deploying osd.9 on smithi031 with /dev/vg_nvme/lv_1... 2024-04-25T13:15:31.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:15:31.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:31 smithi031 ceph-mon[29457]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:31.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:31 smithi031 ceph-mon[29457]: from='osd.8 [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:15:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:31 smithi031 ceph-mon[29457]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:15:31.616 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:15:31.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:31 smithi066 ceph-mon[33137]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:31.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:31.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:31.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:31 smithi066 ceph-mon[33137]: from='osd.8 [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:15:31.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:31 smithi066 ceph-mon[33137]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:15:32.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:32.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:32.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:32 smithi136 ceph-mon[33038]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:15:32.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:32 smithi136 ceph-mon[33038]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:15:32.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:32.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:32 smithi136 ceph-mon[33038]: from='osd.8 [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:15:32.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:32 smithi136 ceph-mon[33038]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:15:32.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:32.755 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:32.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:32 smithi031 ceph-mon[29457]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:15:32.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:32 smithi031 ceph-mon[29457]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:15:32.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:32.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:32 smithi031 ceph-mon[29457]: from='osd.8 [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:15:32.756 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:32 smithi031 ceph-mon[29457]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:15:32.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:32.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:32.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:32 smithi066 ceph-mon[33137]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:15:32.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:32 smithi066 ceph-mon[33137]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:15:32.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:32.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:32 smithi066 ceph-mon[33137]: from='osd.8 [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:15:32.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:32 smithi066 ceph-mon[33137]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:15:33.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:33 smithi136 ceph-mon[33038]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:33.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:33 smithi136 ceph-mon[33038]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:15:33.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:33 smithi136 ceph-mon[33038]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T13:15:33.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:33.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:33.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:33 smithi136 ceph-mon[33038]: from='osd.8 ' entity='osd.8' 2024-04-25T13:15:33.719 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:15:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:15:33.398+0000 7f6ca7e04700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T13:15:33.720 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:15:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:15:33.499+0000 7f6c9fbef700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:15:33.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:33 smithi031 ceph-mon[29457]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:33.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:33 smithi031 ceph-mon[29457]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:15:33.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:33 smithi031 ceph-mon[29457]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T13:15:33.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:33.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:33.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:33 smithi031 ceph-mon[29457]: from='osd.8 ' entity='osd.8' 2024-04-25T13:15:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:15:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:15:33.845 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-25T13:15:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-25T13:15:33.845 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113602 s, 923 MB/s 2024-04-25T13:15:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-25T13:15:33.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:33 smithi066 ceph-mon[33137]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:33.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:33 smithi066 ceph-mon[33137]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:15:33.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:33 smithi066 ceph-mon[33137]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T13:15:33.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:33 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:33.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:33 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:33.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:33 smithi066 ceph-mon[33137]: from='osd.8 ' entity='osd.8' 2024-04-25T13:15:34.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_1 2024-04-25T13:15:34.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:34 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:15:34.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:34 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:15:34.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:34 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:34.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:34 smithi136 ceph-mon[33038]: osd.8 [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141] boot 2024-04-25T13:15:34.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:34 smithi136 ceph-mon[33038]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T13:15:34.825 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:34 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:15:34.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:34 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:15:34.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:34 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:15:34.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:34 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:34.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:34 smithi066 ceph-mon[33137]: osd.8 [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141] boot 2024-04-25T13:15:34.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:34 smithi066 ceph-mon[33137]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T13:15:34.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:34 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:34 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:15:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:34 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:15:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:34 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:34 smithi031 ceph-mon[29457]: osd.8 [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141] boot 2024-04-25T13:15:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:34 smithi031 ceph-mon[29457]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T13:15:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:34 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:35.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:35 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:35.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T13:15:35.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:35.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:35.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:35.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:35 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:35.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:35.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:35.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:35 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.621+0000 7fa3538a9700 1 -- 172.21.15.31:0/961417416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa34c104660 msgr2=0x7fa34c10af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.621+0000 7fa3538a9700 1 --2- 172.21.15.31:0/961417416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa34c104660 0x7fa34c10af10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa34400b0d0 tx=0x7fa34400b3e0 comp rx=0 tx=0).stop 2024-04-25T13:15:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.621+0000 7fa3538a9700 1 -- 172.21.15.31:0/961417416 shutdown_connections 2024-04-25T13:15:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.621+0000 7fa3538a9700 1 --2- 172.21.15.31:0/961417416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa34c104660 0x7fa34c10af10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.621+0000 7fa3538a9700 1 --2- 172.21.15.31:0/961417416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c102ac0 0x7fa34c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.621+0000 7fa3538a9700 1 --2- 172.21.15.31:0/961417416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa34c103cc0 0x7fa34c104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.621+0000 7fa3538a9700 1 -- 172.21.15.31:0/961417416 >> 172.21.15.31:0/961417416 conn(0x7fa34c0fe060 msgr2=0x7fa34c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:36.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.621+0000 7fa3538a9700 1 -- 172.21.15.31:0/961417416 shutdown_connections 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.621+0000 7fa3538a9700 1 -- 172.21.15.31:0/961417416 wait complete. 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.622+0000 7fa3538a9700 1 Processor -- start 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.622+0000 7fa3538a9700 1 -- start start 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.622+0000 7fa3538a9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c102ac0 0x7fa34c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.622+0000 7fa3538a9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa34c103cc0 0x7fa34c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.622+0000 7fa3538a9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa34c104660 0x7fa34c1a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.622+0000 7fa3538a9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c10e590 con 0x7fa34c104660 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.622+0000 7fa3538a9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c10e410 con 0x7fa34c102ac0 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.622+0000 7fa3538a9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c10e710 con 0x7fa34c103cc0 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa351e46700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa34c104660 0x7fa34c1a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa351e46700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa34c104660 0x7fa34c1a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35706/0 (socket says 172.21.15.31:35706) 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa351e46700 1 -- 172.21.15.31:0/1991492860 learned_addr learned my addr 172.21.15.31:0/1991492860 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:15:36.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa351645700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c102ac0 0x7fa34c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa350e44700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa34c103cc0 0x7fa34c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa351e46700 1 -- 172.21.15.31:0/1991492860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa34c103cc0 msgr2=0x7fa34c199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa351e46700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa34c103cc0 0x7fa34c199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa351e46700 1 -- 172.21.15.31:0/1991492860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c102ac0 msgr2=0x7fa34c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa351e46700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c102ac0 0x7fa34c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa351e46700 1 -- 172.21.15.31:0/1991492860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c1a0740 con 0x7fa34c104660 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa351645700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c102ac0 0x7fa34c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa350e44700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa34c103cc0 0x7fa34c199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa351e46700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa34c104660 0x7fa34c1a00a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa344004cb0 tx=0x7fa344004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa3427fc700 1 -- 172.21.15.31:0/1991492860 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34401f030 con 0x7fa34c104660 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa3427fc700 1 -- 172.21.15.31:0/1991492860 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa344007ac0 con 0x7fa34c104660 2024-04-25T13:15:36.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa3538a9700 1 -- 172.21.15.31:0/1991492860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34c1a09d0 con 0x7fa34c104660 2024-04-25T13:15:36.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa3427fc700 1 -- 172.21.15.31:0/1991492860 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34401b690 con 0x7fa34c104660 2024-04-25T13:15:36.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.623+0000 7fa3538a9700 1 -- 172.21.15.31:0/1991492860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34c1a0e80 con 0x7fa34c104660 2024-04-25T13:15:36.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.624+0000 7fa3538a9700 1 -- 172.21.15.31:0/1991492860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa34c069730 con 0x7fa34c104660 2024-04-25T13:15:36.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.625+0000 7fa3427fc700 1 -- 172.21.15.31:0/1991492860 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa344022030 con 0x7fa34c104660 2024-04-25T13:15:36.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.626+0000 7fa3427fc700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa3380a3cc0 0x7fa3380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:15:36.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.626+0000 7fa351645700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa3380a3cc0 0x7fa3380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:15:36.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.626+0000 7fa3427fc700 1 -- 172.21.15.31:0/1991492860 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5858+0+0 (secure 0 0 0) 0x7fa34400f020 con 0x7fa34c104660 2024-04-25T13:15:36.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.627+0000 7fa351645700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa3380a3cc0 0x7fa3380a6180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa348005fd0 tx=0x7fa348005f00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:15:36.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.631+0000 7fa3427fc700 1 -- 172.21.15.31:0/1991492860 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa344029020 con 0x7fa34c104660 2024-04-25T13:15:36.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:36 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi136 2024-04-25T13:15:36.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:36 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi136 to 5696M 2024-04-25T13:15:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:36 smithi031 ceph-mon[29457]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:15:36.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:36.855+0000 7fa3538a9700 1 -- 172.21.15.31:0/1991492860 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa34c062e80 con 0x7fa3380a3cc0 2024-04-25T13:15:36.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:36 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi136 2024-04-25T13:15:36.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:36 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi136 to 5696M 2024-04-25T13:15:36.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:36 smithi066 ceph-mon[33137]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:15:36.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:36 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi136 2024-04-25T13:15:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:36 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi136 to 5696M 2024-04-25T13:15:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:36 smithi136 ceph-mon[33038]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:15:37.788 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:37 smithi031 ceph-mon[29457]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:37.789 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:37 smithi031 ceph-mon[29457]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:15:37.789 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:15:37.789 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:15:37.789 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:37.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:37 smithi066 ceph-mon[33137]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:37.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:37 smithi066 ceph-mon[33137]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:15:37.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:15:37.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:15:37.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:37.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:37 smithi136 ceph-mon[33038]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:37.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:37 smithi136 ceph-mon[33038]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:15:37.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:15:37.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:15:37.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:38.822 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:38 smithi031 ceph-mon[29457]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:15:38.822 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:38 smithi031 ceph-mon[29457]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T13:15:38.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:38 smithi066 ceph-mon[33137]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:15:38.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:38 smithi066 ceph-mon[33137]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T13:15:38.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:38 smithi136 ceph-mon[33038]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:15:38.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:38 smithi136 ceph-mon[33038]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T13:15:39.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:39 smithi066 ceph-mon[33137]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:39.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:39 smithi136 ceph-mon[33038]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:39.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:39 smithi031 ceph-mon[29457]: pgmap v208: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:40.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:40 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/956253940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8974813-c7d4-4aa5-b86f-7c2403d2b29c"}]: dispatch 2024-04-25T13:15:40.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:40 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/956253940' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8974813-c7d4-4aa5-b86f-7c2403d2b29c"}]': finished 2024-04-25T13:15:40.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:40 smithi066 ceph-mon[33137]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T13:15:40.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:40 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:40.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:40 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2015553657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:15:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:40 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/956253940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8974813-c7d4-4aa5-b86f-7c2403d2b29c"}]: dispatch 2024-04-25T13:15:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:40 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/956253940' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8974813-c7d4-4aa5-b86f-7c2403d2b29c"}]': finished 2024-04-25T13:15:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:40 smithi136 ceph-mon[33038]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T13:15:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:40 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:40 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2015553657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:15:40.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:40 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/956253940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8974813-c7d4-4aa5-b86f-7c2403d2b29c"}]: dispatch 2024-04-25T13:15:40.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:40 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/956253940' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8974813-c7d4-4aa5-b86f-7c2403d2b29c"}]': finished 2024-04-25T13:15:40.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:40 smithi031 ceph-mon[29457]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T13:15:40.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:40 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:40.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:40 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2015553657' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:15:41.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:41 smithi136 ceph-mon[33038]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-25T13:15:41.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:41 smithi066 ceph-mon[33137]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-25T13:15:41.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:41 smithi031 ceph-mon[29457]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-25T13:15:42.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:42 smithi066 ceph-mon[33137]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:15:42.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:42 smithi136 ceph-mon[33038]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:15:42.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:42 smithi031 ceph-mon[29457]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:15:44.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:44 smithi031 ceph-mon[29457]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-25T13:15:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:44 smithi066 ceph-mon[33137]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-25T13:15:45.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:44 smithi136 ceph-mon[33038]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-25T13:15:46.940 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:46 smithi031 ceph-mon[29457]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-25T13:15:47.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:46 smithi066 ceph-mon[33137]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-25T13:15:47.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:46 smithi136 ceph-mon[33038]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-25T13:15:47.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:47 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:15:47.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:47 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:48.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:47 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:15:48.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:47 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:48.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:47 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:15:48.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:47 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:48.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:48 smithi031 ceph-mon[29457]: Deploying daemon osd.9 on smithi031 2024-04-25T13:15:48.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:48 smithi031 ceph-mon[29457]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:15:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:48 smithi066 ceph-mon[33137]: Deploying daemon osd.9 on smithi031 2024-04-25T13:15:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:48 smithi066 ceph-mon[33137]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:15:49.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:48 smithi136 ceph-mon[33038]: Deploying daemon osd.9 on smithi031 2024-04-25T13:15:49.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:48 smithi136 ceph-mon[33038]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:15:50.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:50 smithi031 ceph-mon[29457]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:51.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:50 smithi066 ceph-mon[33137]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:51.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:50 smithi136 ceph-mon[33038]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:52 smithi066 ceph-mon[33137]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:53.170 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:52 smithi031 ceph-mon[29457]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:53.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:52 smithi136 ceph-mon[33038]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:54.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:54.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:54.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:54.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:54.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:54.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:55.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:55 smithi031 ceph-mon[29457]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:55.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:55 smithi136 ceph-mon[33038]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:55.896 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:55 smithi066 ceph-mon[33137]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.490+0000 7fa3427fc700 1 -- 172.21.15.31:0/1991492860 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa34c062e80 con 0x7fa3380a3cc0 2024-04-25T13:15:57.490 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 9 on host 'smithi031' 2024-04-25T13:15:57.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.493+0000 7fa3538a9700 1 -- 172.21.15.31:0/1991492860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa3380a3cc0 msgr2=0x7fa3380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.493+0000 7fa3538a9700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa3380a3cc0 0x7fa3380a6180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa348005fd0 tx=0x7fa348005f00 comp rx=0 tx=0).stop 2024-04-25T13:15:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.493+0000 7fa3538a9700 1 -- 172.21.15.31:0/1991492860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa34c104660 msgr2=0x7fa34c1a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:15:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.493+0000 7fa3538a9700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa34c104660 0x7fa34c1a00a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa344004cb0 tx=0x7fa344004d90 comp rx=0 tx=0).stop 2024-04-25T13:15:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.493+0000 7fa3538a9700 1 -- 172.21.15.31:0/1991492860 shutdown_connections 2024-04-25T13:15:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.493+0000 7fa3538a9700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa34c104660 0x7fa34c1a00a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.494+0000 7fa3538a9700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa34c102ac0 0x7fa34c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.494+0000 7fa3538a9700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa3380a3cc0 0x7fa3380a6180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.494+0000 7fa3538a9700 1 --2- 172.21.15.31:0/1991492860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa34c103cc0 0x7fa34c199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:15:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.494+0000 7fa3538a9700 1 -- 172.21.15.31:0/1991492860 >> 172.21.15.31:0/1991492860 conn(0x7fa34c0fe060 msgr2=0x7fa34c106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:15:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.494+0000 7fa3538a9700 1 -- 172.21.15.31:0/1991492860 shutdown_connections 2024-04-25T13:15:57.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:15:57.494+0000 7fa3538a9700 1 -- 172.21.15.31:0/1991492860 wait complete. 2024-04-25T13:15:57.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:57 smithi031 ceph-mon[29457]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:57 smithi031 ceph-mon[29457]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:15:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:57 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:57.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:57 smithi066 ceph-mon[33137]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:57.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:57.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:57.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:57 smithi066 ceph-mon[33137]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:15:57.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:57 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:57.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:57 smithi136 ceph-mon[33038]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:57.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:15:57.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:15:57.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:57 smithi136 ceph-mon[33038]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:15:57.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:15:57.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:57 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:15:58.149 DEBUG:teuthology.orchestra.run.smithi031:osd.9> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T13:15:58.151 INFO:tasks.cephadm:Deploying osd.10 on smithi066 with /dev/vg_nvme/lv_1... 2024-04-25T13:15:58.151 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:15:58.408 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:15:58.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:58 smithi031 ceph-mon[29457]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:15:58.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:58 smithi031 ceph-mon[29457]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T13:15:58.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:58 smithi031 ceph-mon[29457]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:15:58.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:58 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:58.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:58 smithi031 ceph-mon[29457]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:58 smithi066 ceph-mon[33137]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:15:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:58 smithi066 ceph-mon[33137]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T13:15:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:58 smithi066 ceph-mon[33137]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:15:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:58 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:58.818 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:58 smithi066 ceph-mon[33137]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:58.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:58 smithi136 ceph-mon[33038]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:15:58.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:58 smithi136 ceph-mon[33038]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T13:15:58.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:58 smithi136 ceph-mon[33038]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-25T13:15:58.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:58 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:58.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:58 smithi136 ceph-mon[33038]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:15:59.664 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:59 smithi031 ceph-mon[29457]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:15:59.664 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:59 smithi031 ceph-mon[29457]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T13:15:59.664 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:59 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:59.664 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:59 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:59.664 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:59 smithi031 ceph-mon[29457]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' 2024-04-25T13:15:59.664 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:15:59 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:59.665 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:15:59 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:15:59.439+0000 7fbb9a921700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T13:15:59.665 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:15:59 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:15:59.522+0000 7fbb9270c700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:15:59.798 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:59 smithi066 ceph-mon[33137]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:15:59.798 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:59 smithi066 ceph-mon[33137]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T13:15:59.798 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:59 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:59.798 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:59 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:59.798 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:59 smithi066 ceph-mon[33137]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' 2024-04-25T13:15:59.799 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:15:59 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:59.870 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:59 smithi136 ceph-mon[33038]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-25T13:15:59.870 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:59 smithi136 ceph-mon[33038]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T13:15:59.870 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:59 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:59.870 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:59 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:15:59.870 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:59 smithi136 ceph-mon[33038]: from='osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340]' entity='osd.9' 2024-04-25T13:15:59.871 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:15:59 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:16:00.387 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:16:00.387 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:16:00.387 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-04-25T13:16:00.387 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-04-25T13:16:00.387 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114749 s, 914 MB/s 2024-04-25T13:16:00.387 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-04-25T13:16:00.828 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:00 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:16:00.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:00 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:16:00.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:00 smithi066 ceph-mon[33137]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:16:00.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:00 smithi066 ceph-mon[33137]: osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340] boot 2024-04-25T13:16:00.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:00 smithi066 ceph-mon[33137]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T13:16:00.829 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:00 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:16:00.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:00 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:16:00.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:00 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:16:00.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:00 smithi136 ceph-mon[33038]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:16:00.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:00 smithi136 ceph-mon[33038]: osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340] boot 2024-04-25T13:16:00.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:00 smithi136 ceph-mon[33038]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T13:16:00.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:00 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:16:00.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:00 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:16:00.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:00 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:16:00.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:00 smithi031 ceph-mon[29457]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:16:00.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:00 smithi031 ceph-mon[29457]: osd.9 [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340] boot 2024-04-25T13:16:00.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:00 smithi031 ceph-mon[29457]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T13:16:00.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:00 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:16:01.115 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi066:vg_nvme/lv_1 2024-04-25T13:16:01.373 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:16:02.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi031 2024-04-25T13:16:02.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.623 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.624 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.624 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi031 to 4273M 2024-04-25T13:16:02.624 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.624 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:02.624 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:02.624 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.624 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:02 smithi066 ceph-mon[33137]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T13:16:02.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi031 2024-04-25T13:16:02.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi031 to 4273M 2024-04-25T13:16:02.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:02.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:02.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:02 smithi136 ceph-mon[33038]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T13:16:02.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi031 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi031 to 4273M 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:02 smithi031 ceph-mon[29457]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T13:16:02.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.953+0000 7fd5951e7700 1 -- 172.21.15.66:0/980172762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd590100b10 msgr2=0x7fd590100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:02.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.953+0000 7fd5951e7700 1 --2- 172.21.15.66:0/980172762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd590100b10 0x7fd590100f90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd584009990 tx=0x7fd584009ca0 comp rx=0 tx=0).stop 2024-04-25T13:16:02.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.953+0000 7fd5951e7700 1 -- 172.21.15.66:0/980172762 shutdown_connections 2024-04-25T13:16:02.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.953+0000 7fd5951e7700 1 --2- 172.21.15.66:0/980172762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd590103010 0x7fd590103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:02.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.953+0000 7fd5951e7700 1 --2- 172.21.15.66:0/980172762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5901014d0 0x7fd590101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:02.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.953+0000 7fd5951e7700 1 --2- 172.21.15.66:0/980172762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd590100b10 0x7fd590100f90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:02.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.953+0000 7fd5951e7700 1 -- 172.21.15.66:0/980172762 >> 172.21.15.66:0/980172762 conn(0x7fd5900fc6d0 msgr2=0x7fd5900feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:02.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.953+0000 7fd5951e7700 1 -- 172.21.15.66:0/980172762 shutdown_connections 2024-04-25T13:16:02.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.953+0000 7fd5951e7700 1 -- 172.21.15.66:0/980172762 wait complete. 2024-04-25T13:16:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd5951e7700 1 Processor -- start 2024-04-25T13:16:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd5951e7700 1 -- start start 2024-04-25T13:16:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd5951e7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd590100b10 0x7fd59019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd5951e7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5901014d0 0x7fd59019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd5951e7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd590103010 0x7fd5901a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd5951e7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd590110720 con 0x7fd590100b10 2024-04-25T13:16:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd5951e7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901105a0 con 0x7fd5901014d0 2024-04-25T13:16:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd5951e7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901108a0 con 0x7fd590103010 2024-04-25T13:16:02.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd58e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5901014d0 0x7fd59019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd58e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5901014d0 0x7fd59019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60382/0 (socket says 172.21.15.66:60382) 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd58e59c700 1 -- 172.21.15.66:0/4068820253 learned_addr learned my addr 172.21.15.66:0/4068820253 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd58f59e700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd590103010 0x7fd5901a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.954+0000 7fd58ed9d700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd590100b10 0x7fd59019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.955+0000 7fd58e59c700 1 -- 172.21.15.66:0/4068820253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd590103010 msgr2=0x7fd5901a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.955+0000 7fd58e59c700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd590103010 0x7fd5901a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.955+0000 7fd58e59c700 1 -- 172.21.15.66:0/4068820253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd590100b10 msgr2=0x7fd59019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.955+0000 7fd58e59c700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd590100b10 0x7fd59019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.955+0000 7fd58e59c700 1 -- 172.21.15.66:0/4068820253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5901a2a70 con 0x7fd5901014d0 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.955+0000 7fd58e59c700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5901014d0 0x7fd59019b3a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd58400f690 tx=0x7fd58400f6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.955+0000 7fd577fff700 1 -- 172.21.15.66:0/4068820253 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58400fe10 con 0x7fd5901014d0 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.955+0000 7fd5951e7700 1 -- 172.21.15.66:0/4068820253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5901a2d00 con 0x7fd5901014d0 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.955+0000 7fd577fff700 1 -- 172.21.15.66:0/4068820253 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd58400be40 con 0x7fd5901014d0 2024-04-25T13:16:02.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.955+0000 7fd577fff700 1 -- 172.21.15.66:0/4068820253 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584021410 con 0x7fd5901014d0 2024-04-25T13:16:02.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.955+0000 7fd5951e7700 1 -- 172.21.15.66:0/4068820253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5901a30f0 con 0x7fd5901014d0 2024-04-25T13:16:02.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.956+0000 7fd5951e7700 1 -- 172.21.15.66:0/4068820253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd590069730 con 0x7fd5901014d0 2024-04-25T13:16:02.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.957+0000 7fd577fff700 1 -- 172.21.15.66:0/4068820253 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd584009d10 con 0x7fd5901014d0 2024-04-25T13:16:02.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.958+0000 7fd577fff700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5780a3c40 0x7fd5780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:02.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.958+0000 7fd577fff700 1 -- 172.21.15.66:0/4068820253 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7fd5840d0c50 con 0x7fd5901014d0 2024-04-25T13:16:02.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.958+0000 7fd58ed9d700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5780a3c40 0x7fd5780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:02.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.959+0000 7fd58ed9d700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5780a3c40 0x7fd5780a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd580004630 tx=0x7fd58000b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:02.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:02.961+0000 7fd577fff700 1 -- 172.21.15.66:0/4068820253 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd584013070 con 0x7fd5901014d0 2024-04-25T13:16:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:03.181+0000 7fd5951e7700 1 -- 172.21.15.66:0/4068820253 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd590062f40 con 0x7fd5780a3c40 2024-04-25T13:16:03.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:03 smithi066 ceph-mon[33137]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:03.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:03 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:16:03.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:03 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:16:03.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:03 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:03.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:03 smithi136 ceph-mon[33038]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:03.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:03 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:16:03.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:03 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:16:03.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:03 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:03.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:03 smithi031 ceph-mon[29457]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:03.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:03 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:16:03.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:03 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:16:03.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:03 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:04.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:04 smithi066 ceph-mon[33137]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:04.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:04 smithi136 ceph-mon[33038]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:04.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:04 smithi031 ceph-mon[29457]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:05.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:05 smithi066 ceph-mon[33137]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:05.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:05 smithi136 ceph-mon[33038]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:05.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:05 smithi031 ceph-mon[29457]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:06.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:06 smithi066 ceph-mon[33137]: from='client.? 172.21.15.66:0/1759141068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b500a9-fd99-46fd-8405-c10168928b31"}]: dispatch 2024-04-25T13:16:06.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:06 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b500a9-fd99-46fd-8405-c10168928b31"}]: dispatch 2024-04-25T13:16:06.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:06 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b500a9-fd99-46fd-8405-c10168928b31"}]': finished 2024-04-25T13:16:06.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:06 smithi066 ceph-mon[33137]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T13:16:06.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:06 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:06.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:06 smithi136 ceph-mon[33038]: from='client.? 172.21.15.66:0/1759141068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b500a9-fd99-46fd-8405-c10168928b31"}]: dispatch 2024-04-25T13:16:06.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:06 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b500a9-fd99-46fd-8405-c10168928b31"}]: dispatch 2024-04-25T13:16:06.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:06 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b500a9-fd99-46fd-8405-c10168928b31"}]': finished 2024-04-25T13:16:06.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:06 smithi136 ceph-mon[33038]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T13:16:06.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:06 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:06.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:06 smithi031 ceph-mon[29457]: from='client.? 172.21.15.66:0/1759141068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b500a9-fd99-46fd-8405-c10168928b31"}]: dispatch 2024-04-25T13:16:06.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:06 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b500a9-fd99-46fd-8405-c10168928b31"}]: dispatch 2024-04-25T13:16:06.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:06 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b500a9-fd99-46fd-8405-c10168928b31"}]': finished 2024-04-25T13:16:06.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:06 smithi031 ceph-mon[29457]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T13:16:06.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:06 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:07.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:07 smithi066 ceph-mon[33137]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:07.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:07 smithi066 ceph-mon[33137]: from='client.? 172.21.15.66:0/1351720734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:16:07.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:07 smithi136 ceph-mon[33038]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:07.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:07 smithi136 ceph-mon[33038]: from='client.? 172.21.15.66:0/1351720734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:16:07.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:07 smithi031 ceph-mon[29457]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:07.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:07 smithi031 ceph-mon[29457]: from='client.? 172.21.15.66:0/1351720734' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:16:09.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:09 smithi066 ceph-mon[33137]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:09.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:09 smithi136 ceph-mon[33038]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:09.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:09 smithi031 ceph-mon[29457]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:11.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:11 smithi136 ceph-mon[33038]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:11.492 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:11 smithi066 ceph-mon[33137]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:11.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:11 smithi031 ceph-mon[29457]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:13.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:13 smithi066 ceph-mon[33137]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:13.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:13 smithi136 ceph-mon[33038]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:13.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:13 smithi031 ceph-mon[29457]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:14.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:16:14.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:14.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:14 smithi066 ceph-mon[33137]: Deploying daemon osd.10 on smithi066 2024-04-25T13:16:14.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:16:14.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:14.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:14 smithi136 ceph-mon[33038]: Deploying daemon osd.10 on smithi066 2024-04-25T13:16:14.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:16:14.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:14.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:14 smithi031 ceph-mon[29457]: Deploying daemon osd.10 on smithi066 2024-04-25T13:16:15.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:15 smithi066 ceph-mon[33137]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:15.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:15 smithi136 ceph-mon[33038]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:15.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:15 smithi031 ceph-mon[29457]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:17.424 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:17 smithi066 ceph-mon[33137]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:17.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:17 smithi136 ceph-mon[33038]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:17.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:17 smithi031 ceph-mon[29457]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:19.657 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:19 smithi066 ceph-mon[33137]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:19.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:19.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:19.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:19.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:19 smithi136 ceph-mon[33038]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:19.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:19.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:19.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:19.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:19 smithi031 ceph-mon[29457]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:19.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:19.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:19.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:21.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:21 smithi066 ceph-mon[33137]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:21.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:21 smithi136 ceph-mon[33038]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:21.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:21 smithi031 ceph-mon[29457]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:22.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.951+0000 7fd577fff700 1 -- 172.21.15.66:0/4068820253 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd590062f40 con 0x7fd5780a3c40 2024-04-25T13:16:22.951 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 10 on host 'smithi066' 2024-04-25T13:16:22.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.953+0000 7fd5951e7700 1 -- 172.21.15.66:0/4068820253 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5780a3c40 msgr2=0x7fd5780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:22.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.953+0000 7fd5951e7700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5780a3c40 0x7fd5780a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd580004630 tx=0x7fd58000b3c0 comp rx=0 tx=0).stop 2024-04-25T13:16:22.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.953+0000 7fd5951e7700 1 -- 172.21.15.66:0/4068820253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5901014d0 msgr2=0x7fd59019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:22.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.953+0000 7fd5951e7700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5901014d0 0x7fd59019b3a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd58400f690 tx=0x7fd58400f6c0 comp rx=0 tx=0).stop 2024-04-25T13:16:22.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.954+0000 7fd5951e7700 1 -- 172.21.15.66:0/4068820253 shutdown_connections 2024-04-25T13:16:22.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.954+0000 7fd5951e7700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5780a3c40 0x7fd5780a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:22.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.954+0000 7fd5951e7700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd590103010 0x7fd5901a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:22.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.954+0000 7fd5951e7700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd590100b10 0x7fd59019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:22.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.954+0000 7fd5951e7700 1 --2- 172.21.15.66:0/4068820253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5901014d0 0x7fd59019b3a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:22.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.954+0000 7fd5951e7700 1 -- 172.21.15.66:0/4068820253 >> 172.21.15.66:0/4068820253 conn(0x7fd5900fc6d0 msgr2=0x7fd5900fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:22.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.954+0000 7fd5951e7700 1 -- 172.21.15.66:0/4068820253 shutdown_connections 2024-04-25T13:16:22.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-04-25T13:16:22.954+0000 7fd5951e7700 1 -- 172.21.15.66:0/4068820253 wait complete. 2024-04-25T13:16:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:23 smithi066 ceph-mon[33137]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:23 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:23 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:23 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:23 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:23 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:23 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:23 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:23 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:23.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:23 smithi066 ceph-mon[33137]: from='osd.10 [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:16:23.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:23 smithi066 ceph-mon[33137]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:16:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:23 smithi136 ceph-mon[33038]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:23 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:23 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:23 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:23 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:23 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:23 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:23 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:23 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:23 smithi136 ceph-mon[33038]: from='osd.10 [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:16:23.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:23 smithi136 ceph-mon[33038]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:16:23.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:23 smithi031 ceph-mon[29457]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:23.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:23 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:23 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:23 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:23 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:23 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:23 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:23 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:23 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:23 smithi031 ceph-mon[29457]: from='osd.10 [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:16:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:23 smithi031 ceph-mon[29457]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:16:23.628 DEBUG:teuthology.orchestra.run.smithi066:osd.10> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T13:16:23.630 INFO:tasks.cephadm:Deploying osd.11 on smithi136 with /dev/vg_nvme/lv_1... 2024-04-25T13:16:23.631 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:16:23.877 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:16:25.061 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:24 smithi066 ceph-mon[33137]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:25.061 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:24 smithi066 ceph-mon[33137]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:16:25.061 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:24 smithi066 ceph-mon[33137]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T13:16:25.061 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:24 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:25.062 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:24 smithi066 ceph-mon[33137]: from='osd.10 [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:16:25.062 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:24 smithi066 ceph-mon[33137]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:16:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:24 smithi136 ceph-mon[33038]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:24 smithi136 ceph-mon[33038]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:16:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:24 smithi136 ceph-mon[33038]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T13:16:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:24 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:25.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:24 smithi136 ceph-mon[33038]: from='osd.10 [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:16:25.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:24 smithi136 ceph-mon[33038]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:16:25.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:24 smithi031 ceph-mon[29457]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:25.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:24 smithi031 ceph-mon[29457]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:16:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:24 smithi031 ceph-mon[29457]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T13:16:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:24 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:24 smithi031 ceph-mon[29457]: from='osd.10 [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:16:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:24 smithi031 ceph-mon[29457]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-04-25T13:16:25.947 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:16:25.947 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:16:25.947 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-04-25T13:16:25.947 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-04-25T13:16:25.947 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00913095 s, 1.1 GB/s 2024-04-25T13:16:25.947 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-04-25T13:16:26.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:25 smithi066 ceph-mon[33137]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:16:26.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:25 smithi066 ceph-mon[33137]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T13:16:26.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:25 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:26.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:25 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:26.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:25 smithi066 ceph-mon[33137]: from='osd.10 ' entity='osd.10' 2024-04-25T13:16:26.158 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:16:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:16:25.864+0000 7ff0f8a51700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T13:16:26.158 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:16:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:16:25.958+0000 7ff0f003b700 -1 osd.10 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:16:26.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:25 smithi136 ceph-mon[33038]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:16:26.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:25 smithi136 ceph-mon[33038]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T13:16:26.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:25 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:26.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:25 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:26.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:25 smithi136 ceph-mon[33038]: from='osd.10 ' entity='osd.10' 2024-04-25T13:16:26.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:25 smithi031 ceph-mon[29457]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-04-25T13:16:26.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:25 smithi031 ceph-mon[29457]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T13:16:26.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:25 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:26.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:25 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:26.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:25 smithi031 ceph-mon[29457]: from='osd.10 ' entity='osd.10' 2024-04-25T13:16:26.669 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_1 2024-04-25T13:16:26.917 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:16:27.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:16:27.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:16:27.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:27.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T13:16:27.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:27.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:27.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:27.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:26 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:27.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:16:27.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:16:27.208 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:26 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:27.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:16:27.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:26 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:28.173 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:27 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi066 2024-04-25T13:16:28.173 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:27 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi066 to 4272M 2024-04-25T13:16:28.173 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:27 smithi136 ceph-mon[33038]: osd.10 [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636] boot 2024-04-25T13:16:28.174 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:27 smithi136 ceph-mon[33038]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T13:16:28.174 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:28.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:27 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi066 2024-04-25T13:16:28.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:27 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi066 to 4272M 2024-04-25T13:16:28.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:27 smithi031 ceph-mon[29457]: osd.10 [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636] boot 2024-04-25T13:16:28.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:27 smithi031 ceph-mon[29457]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T13:16:28.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:28.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:27 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi066 2024-04-25T13:16:28.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:27 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi066 to 4272M 2024-04-25T13:16:28.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:27 smithi066 ceph-mon[33137]: osd.10 [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636] boot 2024-04-25T13:16:28.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:27 smithi066 ceph-mon[33137]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T13:16:28.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:16:28.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.535+0000 7f4283142700 1 -- 172.21.15.136:0/2758855368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c103ca0 msgr2=0x7f427c104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:28.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.535+0000 7f4283142700 1 --2- 172.21.15.136:0/2758855368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c103ca0 0x7f427c104100 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4268009990 tx=0x7f4268009ca0 comp rx=0 tx=0).stop 2024-04-25T13:16:28.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.535+0000 7f4283142700 1 -- 172.21.15.136:0/2758855368 shutdown_connections 2024-04-25T13:16:28.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.535+0000 7f4283142700 1 --2- 172.21.15.136:0/2758855368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f427c104640 0x7f427c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:28.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.535+0000 7f4283142700 1 --2- 172.21.15.136:0/2758855368 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c102aa0 0x7f427c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:28.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.535+0000 7f4283142700 1 --2- 172.21.15.136:0/2758855368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c103ca0 0x7f427c104100 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:28.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.535+0000 7f4283142700 1 -- 172.21.15.136:0/2758855368 >> 172.21.15.136:0/2758855368 conn(0x7f427c0fe040 msgr2=0x7f427c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:28.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.535+0000 7f4283142700 1 -- 172.21.15.136:0/2758855368 shutdown_connections 2024-04-25T13:16:28.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.535+0000 7f4283142700 1 -- 172.21.15.136:0/2758855368 wait complete. 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f4283142700 1 Processor -- start 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f4283142700 1 -- start start 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f4283142700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c102aa0 0x7f427c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f4283142700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f427c103ca0 0x7f427c199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f4283142700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c104640 0x7f427c1a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f4283142700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f427c10e510 con 0x7f427c103ca0 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f4283142700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f427c10e390 con 0x7f427c102aa0 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f4283142700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f427c10e690 con 0x7f427c104640 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f42816df700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c104640 0x7f427c1a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f42816df700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c104640 0x7f427c1a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:35576/0 (socket says 172.21.15.136:35576) 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f42816df700 1 -- 172.21.15.136:0/3636774367 learned_addr learned my addr 172.21.15.136:0/3636774367 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.536+0000 7f427bfff700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f427c103ca0 0x7f427c199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:28.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f4280ede700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c102aa0 0x7f427c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:28.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f42816df700 1 -- 172.21.15.136:0/3636774367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c102aa0 msgr2=0x7f427c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:28.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f42816df700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c102aa0 0x7f427c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:28.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f42816df700 1 -- 172.21.15.136:0/3636774367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f427c103ca0 msgr2=0x7f427c199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:28.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f42816df700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f427c103ca0 0x7f427c199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:28.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f42816df700 1 -- 172.21.15.136:0/3636774367 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c1a07b0 con 0x7f427c104640 2024-04-25T13:16:28.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f4280ede700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c102aa0 0x7f427c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:16:28.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f42816df700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c104640 0x7f427c1a0110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4274007d40 tx=0x7f427400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:28.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f4279ffb700 1 -- 172.21.15.136:0/3636774367 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274016020 con 0x7f427c104640 2024-04-25T13:16:28.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f4283142700 1 -- 172.21.15.136:0/3636774367 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c1a0aa0 con 0x7f427c104640 2024-04-25T13:16:28.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f4279ffb700 1 -- 172.21.15.136:0/3636774367 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427400ad60 con 0x7f427c104640 2024-04-25T13:16:28.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f4279ffb700 1 -- 172.21.15.136:0/3636774367 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42740116f0 con 0x7f427c104640 2024-04-25T13:16:28.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.537+0000 7f4283142700 1 -- 172.21.15.136:0/3636774367 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c1a0fe0 con 0x7f427c104640 2024-04-25T13:16:28.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.538+0000 7f4283142700 1 -- 172.21.15.136:0/3636774367 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427c069730 con 0x7f427c104640 2024-04-25T13:16:28.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.539+0000 7f4279ffb700 1 -- 172.21.15.136:0/3636774367 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4274019020 con 0x7f427c104640 2024-04-25T13:16:28.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.540+0000 7f4279ffb700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f426c0a3c40 0x7f426c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:28.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.540+0000 7f4279ffb700 1 -- 172.21.15.136:0/3636774367 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6723+0+0 (secure 0 0 0) 0x7f42740cfeb0 con 0x7f427c104640 2024-04-25T13:16:28.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.540+0000 7f4280ede700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f426c0a3c40 0x7f426c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:28.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.540+0000 7f4280ede700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f426c0a3c40 0x7f426c0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4270005fd0 tx=0x7f4270005dc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:28.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.544+0000 7f4279ffb700 1 -- 172.21.15.136:0/3636774367 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4274020020 con 0x7f427c104640 2024-04-25T13:16:28.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:28.748+0000 7f4283142700 1 -- 172.21.15.136:0/3636774367 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f427c062e80 con 0x7f426c0a3c40 2024-04-25T13:16:29.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:28 smithi136 ceph-mon[33038]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:29.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:28 smithi136 ceph-mon[33038]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:16:29.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:28 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:16:29.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:28 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:16:29.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:28 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:29.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:28 smithi031 ceph-mon[29457]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:29.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:28 smithi031 ceph-mon[29457]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:16:29.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:28 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:16:29.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:28 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:16:29.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:28 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:29.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:28 smithi066 ceph-mon[33137]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:16:29.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:28 smithi066 ceph-mon[33137]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:16:29.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:28 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:16:29.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:28 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:16:29.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:28 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:30.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:29 smithi136 ceph-mon[33038]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:30.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:29 smithi136 ceph-mon[33038]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:16:30.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:29 smithi136 ceph-mon[33038]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:16:30.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:29 smithi031 ceph-mon[29457]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:30.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:29 smithi031 ceph-mon[29457]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:16:30.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:29 smithi031 ceph-mon[29457]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:16:30.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:29 smithi066 ceph-mon[33137]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:30.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:29 smithi066 ceph-mon[33137]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:16:30.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:29 smithi066 ceph-mon[33137]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:16:31.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:30 smithi031 ceph-mon[29457]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:31.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:30 smithi031 ceph-mon[29457]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T13:16:31.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:30 smithi066 ceph-mon[33137]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:30 smithi066 ceph-mon[33137]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T13:16:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:30 smithi136 ceph-mon[33038]: pgmap v247: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:30 smithi136 ceph-mon[33038]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T13:16:32.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:31 smithi031 ceph-mon[29457]: from='client.? 172.21.15.136:0/1279899475' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc059f09-0557-40a5-a573-5c304b9bdc25"}]: dispatch 2024-04-25T13:16:32.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:31 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc059f09-0557-40a5-a573-5c304b9bdc25"}]: dispatch 2024-04-25T13:16:32.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:31 smithi031 ceph-mon[29457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc059f09-0557-40a5-a573-5c304b9bdc25"}]': finished 2024-04-25T13:16:32.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:31 smithi031 ceph-mon[29457]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:16:32.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:31 smithi066 ceph-mon[33137]: from='client.? 172.21.15.136:0/1279899475' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc059f09-0557-40a5-a573-5c304b9bdc25"}]: dispatch 2024-04-25T13:16:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:31 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc059f09-0557-40a5-a573-5c304b9bdc25"}]: dispatch 2024-04-25T13:16:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:31 smithi066 ceph-mon[33137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc059f09-0557-40a5-a573-5c304b9bdc25"}]': finished 2024-04-25T13:16:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:31 smithi066 ceph-mon[33137]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:16:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:32.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:31 smithi136 ceph-mon[33038]: from='client.? 172.21.15.136:0/1279899475' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc059f09-0557-40a5-a573-5c304b9bdc25"}]: dispatch 2024-04-25T13:16:32.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:31 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc059f09-0557-40a5-a573-5c304b9bdc25"}]: dispatch 2024-04-25T13:16:32.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:31 smithi136 ceph-mon[33038]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc059f09-0557-40a5-a573-5c304b9bdc25"}]': finished 2024-04-25T13:16:32.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:31 smithi136 ceph-mon[33038]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T13:16:32.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:33.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:32 smithi031 ceph-mon[29457]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:33.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:32 smithi031 ceph-mon[29457]: from='client.? 172.21.15.136:0/4017364142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:16:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:32 smithi066 ceph-mon[33137]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:32 smithi066 ceph-mon[33137]: from='client.? 172.21.15.136:0/4017364142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:16:33.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:32 smithi136 ceph-mon[33038]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:32 smithi136 ceph-mon[33038]: from='client.? 172.21.15.136:0/4017364142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:16:35.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:34 smithi031 ceph-mon[29457]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-25T13:16:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:34 smithi066 ceph-mon[33137]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-25T13:16:35.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:34 smithi136 ceph-mon[33038]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-25T13:16:37.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:36 smithi031 ceph-mon[29457]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:16:37.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:36 smithi136 ceph-mon[33038]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:16:37.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:36 smithi066 ceph-mon[33137]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T13:16:39.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:38 smithi066 ceph-mon[33137]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:16:39.438 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:38 smithi136 ceph-mon[33038]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:16:39.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:38 smithi031 ceph-mon[29457]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:16:40.307 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:16:40.307 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:40.307 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:39 smithi136 ceph-mon[33038]: Deploying daemon osd.11 on smithi136 2024-04-25T13:16:40.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:16:40.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:40.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:39 smithi066 ceph-mon[33137]: Deploying daemon osd.11 on smithi136 2024-04-25T13:16:40.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:16:40.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:40.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:39 smithi031 ceph-mon[29457]: Deploying daemon osd.11 on smithi136 2024-04-25T13:16:41.359 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:40 smithi136 ceph-mon[33038]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:16:41.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:40 smithi066 ceph-mon[33137]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:16:41.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:40 smithi031 ceph-mon[29457]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T13:16:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:42 smithi136 ceph-mon[33038]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-25T13:16:43.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:42 smithi066 ceph-mon[33137]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-25T13:16:43.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:42 smithi031 ceph-mon[29457]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-25T13:16:45.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:44 smithi136 ceph-mon[33038]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:16:45.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:44 smithi066 ceph-mon[33137]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:16:45.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:44 smithi031 ceph-mon[29457]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:16:46.497 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:46.497 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:46.497 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:46 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:46.717 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:46.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:46.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:46 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:46.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:46.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:46.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:46 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:47.537 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:47 smithi136 ceph-mon[33038]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:47.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:47 smithi066 ceph-mon[33137]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:47.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:47 smithi031 ceph-mon[29457]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:49.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.069+0000 7f4279ffb700 1 -- 172.21.15.136:0/3636774367 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f427c062e80 con 0x7f426c0a3c40 2024-04-25T13:16:49.070 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 11 on host 'smithi136' 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 -- 172.21.15.136:0/3636774367 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f426c0a3c40 msgr2=0x7f426c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f426c0a3c40 0x7f426c0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4270005fd0 tx=0x7f4270005dc0 comp rx=0 tx=0).stop 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 -- 172.21.15.136:0/3636774367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c104640 msgr2=0x7f427c1a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c104640 0x7f427c1a0110 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4274007d40 tx=0x7f427400a340 comp rx=0 tx=0).stop 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 -- 172.21.15.136:0/3636774367 shutdown_connections 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f426c0a3c40 0x7f426c0a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c104640 0x7f427c1a0110 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c102aa0 0x7f427c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 --2- 172.21.15.136:0/3636774367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f427c103ca0 0x7f427c199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 -- 172.21.15.136:0/3636774367 >> 172.21.15.136:0/3636774367 conn(0x7f427c0fe040 msgr2=0x7f427c107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 -- 172.21.15.136:0/3636774367 shutdown_connections 2024-04-25T13:16:49.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:16:49.071+0000 7f4283142700 1 -- 172.21.15.136:0/3636774367 wait complete. 2024-04-25T13:16:49.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:49 smithi066 ceph-mon[33137]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:49.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:49 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:49 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:49 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:49.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:49 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:49.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:49 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:49 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:49 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:49 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:49.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:49 smithi066 ceph-mon[33137]: from='osd.11 [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:16:49.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:49 smithi066 ceph-mon[33137]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:16:49.665 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:49 smithi136 ceph-mon[33038]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:49.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:49 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:49 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:49 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:49.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:49 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:49.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:49 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:49 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:49 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:49 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:49.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:49 smithi136 ceph-mon[33038]: from='osd.11 [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:16:49.666 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:49 smithi136 ceph-mon[33038]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:16:49.692 DEBUG:teuthology.orchestra.run.smithi136:osd.11> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T13:16:49.695 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T13:16:49.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T13:16:49.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:49 smithi031 ceph-mon[29457]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:49.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:49 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:49 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:49 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:49.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:49 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:49.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:49 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:49 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:49 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:49.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:49 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:49.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:49 smithi031 ceph-mon[29457]: from='osd.11 [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:16:49.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:49 smithi031 ceph-mon[29457]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:16:49.954 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:16:51.184 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:51 smithi136 ceph-mon[33038]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:51.184 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:51 smithi136 ceph-mon[33038]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:16:51.184 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:51 smithi136 ceph-mon[33038]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:16:51.184 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:51 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:51.184 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:51 smithi136 ceph-mon[33038]: from='osd.11 [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:16:51.184 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:51 smithi136 ceph-mon[33038]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:16:51.185 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:16:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:16:51.050+0000 7f96eb0eb700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T13:16:51.185 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:16:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:16:51.070+0000 7f96e6ede700 -1 osd.11 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:16:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:51 smithi066 ceph-mon[33137]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:51 smithi066 ceph-mon[33137]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:16:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:51 smithi066 ceph-mon[33137]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:16:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:51 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:51 smithi066 ceph-mon[33137]: from='osd.11 [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:16:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:51 smithi066 ceph-mon[33137]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:16:51.424 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:51 smithi031 ceph-mon[29457]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:51.424 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:51 smithi031 ceph-mon[29457]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:16:51.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:51 smithi031 ceph-mon[29457]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:16:51.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:51 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:51.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:51 smithi031 ceph-mon[29457]: from='osd.11 [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:16:51.425 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:51 smithi031 ceph-mon[29457]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T13:16:52.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.095+0000 7fe36ffc2700 1 -- 172.21.15.31:0/3554992545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe368102ba0 msgr2=0x7fe368103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:52.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.095+0000 7fe36ffc2700 1 --2- 172.21.15.31:0/3554992545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe368102ba0 0x7fe368103000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe3580099c0 tx=0x7fe358009cd0 comp rx=0 tx=0).stop 2024-04-25T13:16:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.095+0000 7fe36ffc2700 1 -- 172.21.15.31:0/3554992545 shutdown_connections 2024-04-25T13:16:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.095+0000 7fe36ffc2700 1 --2- 172.21.15.31:0/3554992545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3681019a0 0x7fe368101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.095+0000 7fe36ffc2700 1 --2- 172.21.15.31:0/3554992545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe368103540 0x7fe368109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.095+0000 7fe36ffc2700 1 --2- 172.21.15.31:0/3554992545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe368102ba0 0x7fe368103000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.095+0000 7fe36ffc2700 1 -- 172.21.15.31:0/3554992545 >> 172.21.15.31:0/3554992545 conn(0x7fe3680fcf60 msgr2=0x7fe3680ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36ffc2700 1 -- 172.21.15.31:0/3554992545 shutdown_connections 2024-04-25T13:16:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36ffc2700 1 -- 172.21.15.31:0/3554992545 wait complete. 2024-04-25T13:16:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36ffc2700 1 Processor -- start 2024-04-25T13:16:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36ffc2700 1 -- start start 2024-04-25T13:16:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36ffc2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3681019a0 0x7fe368105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36ffc2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe368102ba0 0x7fe368103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36ffc2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe368103540 0x7fe3681044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36ffc2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe36810d410 con 0x7fe3681019a0 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36ffc2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe36810d290 con 0x7fe368102ba0 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36ffc2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe36810d590 con 0x7fe368103540 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36dd5e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3681019a0 0x7fe368105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36dd5e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3681019a0 0x7fe368105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46228/0 (socket says 172.21.15.31:46228) 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.096+0000 7fe36dd5e700 1 -- 172.21.15.31:0/2740148346 learned_addr learned my addr 172.21.15.31:0/2740148346 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe36e55f700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe368103540 0x7fe3681044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe36d55d700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe368102ba0 0x7fe368103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe36dd5e700 1 -- 172.21.15.31:0/2740148346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe368103540 msgr2=0x7fe3681044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe36dd5e700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe368103540 0x7fe3681044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe36dd5e700 1 -- 172.21.15.31:0/2740148346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe368102ba0 msgr2=0x7fe368103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe36dd5e700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe368102ba0 0x7fe368103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe36dd5e700 1 -- 172.21.15.31:0/2740148346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe368104d70 con 0x7fe3681019a0 2024-04-25T13:16:52.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe36e55f700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe368103540 0x7fe3681044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:16:52.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe36dd5e700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3681019a0 0x7fe368105930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe36400d980 tx=0x7fe36400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:52.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe35effd700 1 -- 172.21.15.31:0/2740148346 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364016070 con 0x7fe3681019a0 2024-04-25T13:16:52.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe35effd700 1 -- 172.21.15.31:0/2740148346 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36400fd40 con 0x7fe3681019a0 2024-04-25T13:16:52.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.097+0000 7fe36ffc2700 1 -- 172.21.15.31:0/2740148346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3681a2d10 con 0x7fe3681019a0 2024-04-25T13:16:52.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.098+0000 7fe35effd700 1 -- 172.21.15.31:0/2740148346 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364019360 con 0x7fe3681019a0 2024-04-25T13:16:52.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.098+0000 7fe36ffc2700 1 -- 172.21.15.31:0/2740148346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3681a31a0 con 0x7fe3681019a0 2024-04-25T13:16:52.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.098+0000 7fe36ffc2700 1 -- 172.21.15.31:0/2740148346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3681a3430 con 0x7fe3681019a0 2024-04-25T13:16:52.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.099+0000 7fe35effd700 1 -- 172.21.15.31:0/2740148346 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe364019070 con 0x7fe3681019a0 2024-04-25T13:16:52.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.100+0000 7fe35effd700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe3540a3cc0 0x7fe3540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:52.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.100+0000 7fe36d55d700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe3540a3cc0 0x7fe3540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:52.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.101+0000 7fe35effd700 1 -- 172.21.15.31:0/2740148346 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe3640cf5b0 con 0x7fe3681019a0 2024-04-25T13:16:52.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.101+0000 7fe36d55d700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe3540a3cc0 0x7fe3540a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe358000c00 tx=0x7fe358005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:52.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.105+0000 7fe35effd700 1 -- 172.21.15.31:0/2740148346 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe36409c430 con 0x7fe3681019a0 2024-04-25T13:16:52.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.313+0000 7fe36ffc2700 1 -- 172.21.15.31:0/2740148346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe3681a3770 con 0x7fe3681019a0 2024-04-25T13:16:52.315 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:52 smithi031 ceph-mon[29457]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:16:52.315 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:52 smithi031 ceph-mon[29457]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T13:16:52.316 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:52.316 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:52 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:52.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.314+0000 7fe35effd700 1 -- 172.21.15.31:0/2740148346 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7fe36409bd30 con 0x7fe3681019a0 2024-04-25T13:16:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:16:52.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 -- 172.21.15.31:0/2740148346 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe3540a3cc0 msgr2=0x7fe3540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe3540a3cc0 0x7fe3540a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe358000c00 tx=0x7fe358005fb0 comp rx=0 tx=0).stop 2024-04-25T13:16:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 -- 172.21.15.31:0/2740148346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3681019a0 msgr2=0x7fe368105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3681019a0 0x7fe368105930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe36400d980 tx=0x7fe36400dc90 comp rx=0 tx=0).stop 2024-04-25T13:16:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 -- 172.21.15.31:0/2740148346 shutdown_connections 2024-04-25T13:16:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe3540a3cc0 0x7fe3540a6180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:52.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe368103540 0x7fe3681044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:52.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3681019a0 0x7fe368105930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:52.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 --2- 172.21.15.31:0/2740148346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe368102ba0 0x7fe368103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:52.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 -- 172.21.15.31:0/2740148346 >> 172.21.15.31:0/2740148346 conn(0x7fe3680fcf60 msgr2=0x7fe3680fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:52.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 -- 172.21.15.31:0/2740148346 shutdown_connections 2024-04-25T13:16:52.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:52.316+0000 7fe36ffc2700 1 -- 172.21.15.31:0/2740148346 wait complete. 2024-04-25T13:16:52.362 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:52 smithi136 ceph-mon[33038]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:16:52.362 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:52 smithi136 ceph-mon[33038]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T13:16:52.363 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:52.363 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:52 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:52.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:52 smithi066 ceph-mon[33137]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T13:16:52.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:52 smithi066 ceph-mon[33137]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T13:16:52.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:52.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:52 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:52.983 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1714051012,"num_in_osds":12,"osd_in_since":1714050991,"num_remapped_pgs":0} 2024-04-25T13:16:52.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:16:53.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: purged_snaps scrub starts 2024-04-25T13:16:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: purged_snaps scrub ok 2024-04-25T13:16:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: osd.11 [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941] boot 2024-04-25T13:16:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:16:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2740148346' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:16:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:53.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:53.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:53.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:53 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:53.275 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:16:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: purged_snaps scrub starts 2024-04-25T13:16:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: purged_snaps scrub ok 2024-04-25T13:16:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: osd.11 [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941] boot 2024-04-25T13:16:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:16:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2740148346' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:16:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:53.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:53.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:53.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:53.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:53 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:53.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: purged_snaps scrub starts 2024-04-25T13:16:53.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: purged_snaps scrub ok 2024-04-25T13:16:53.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:16:53.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: osd.11 [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941] boot 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2740148346' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:16:53.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:53 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:16:54.411 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:54 smithi066 ceph-mon[33137]: Detected new or changed devices on smithi136 2024-04-25T13:16:54.411 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:54 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi136 to 4272M 2024-04-25T13:16:54.411 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:54 smithi066 ceph-mon[33137]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:16:54.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:54 smithi136 ceph-mon[33038]: Detected new or changed devices on smithi136 2024-04-25T13:16:54.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:54 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi136 to 4272M 2024-04-25T13:16:54.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:54 smithi136 ceph-mon[33038]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:16:54.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:54 smithi031 ceph-mon[29457]: Detected new or changed devices on smithi136 2024-04-25T13:16:54.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:54 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi136 to 4272M 2024-04-25T13:16:54.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:54 smithi031 ceph-mon[29457]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:16:55.171 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:55 smithi031 ceph-mon[29457]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:16:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.281+0000 7f0c7950a700 1 -- 172.21.15.31:0/1562429054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c74101400 msgr2=0x7f0c74101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.281+0000 7f0c7950a700 1 --2- 172.21.15.31:0/1562429054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c74101400 0x7f0c74101880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009990 tx=0x7f0c68009ca0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.281+0000 7f0c7950a700 1 -- 172.21.15.31:0/1562429054 shutdown_connections 2024-04-25T13:16:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.281+0000 7f0c7950a700 1 --2- 172.21.15.31:0/1562429054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c74101dc0 0x7f0c7410f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.281+0000 7f0c7950a700 1 --2- 172.21.15.31:0/1562429054 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c74107b20 0x7f0c74107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.281+0000 7f0c7950a700 1 --2- 172.21.15.31:0/1562429054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c74101400 0x7f0c74101880 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.281+0000 7f0c7950a700 1 -- 172.21.15.31:0/1562429054 >> 172.21.15.31:0/1562429054 conn(0x7f0c74076c60 msgr2=0x7f0c74077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.281+0000 7f0c7950a700 1 -- 172.21.15.31:0/1562429054 shutdown_connections 2024-04-25T13:16:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.281+0000 7f0c7950a700 1 -- 172.21.15.31:0/1562429054 wait complete. 2024-04-25T13:16:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.282+0000 7f0c7950a700 1 Processor -- start 2024-04-25T13:16:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.282+0000 7f0c7950a700 1 -- start start 2024-04-25T13:16:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.282+0000 7f0c7950a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c74101400 0x7f0c74199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.282+0000 7f0c7950a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c74101dc0 0x7f0c74199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.282+0000 7f0c7950a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c74107b20 0x7f0c7419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.282+0000 7f0c7950a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0c74112cb0 con 0x7f0c74107b20 2024-04-25T13:16:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c7950a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c74112b30 con 0x7f0c74101400 2024-04-25T13:16:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c7950a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0c74112e30 con 0x7f0c74101dc0 2024-04-25T13:16:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c74107b20 0x7f0c7419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c74107b20 0x7f0c7419d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46256/0 (socket says 172.21.15.31:46256) 2024-04-25T13:16:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c737fe700 1 -- 172.21.15.31:0/426270667 learned_addr learned my addr 172.21.15.31:0/426270667 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:16:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c727fc700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c74101dc0 0x7f0c74199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c72ffd700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c74101400 0x7f0c74199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c737fe700 1 -- 172.21.15.31:0/426270667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c74101dc0 msgr2=0x7f0c74199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:55.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c737fe700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c74101dc0 0x7f0c74199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c737fe700 1 -- 172.21.15.31:0/426270667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c74101400 msgr2=0x7f0c74199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:55.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c737fe700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c74101400 0x7f0c74199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c737fe700 1 -- 172.21.15.31:0/426270667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7419e130 con 0x7f0c74107b20 2024-04-25T13:16:55.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c727fc700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c74101dc0 0x7f0c74199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:16:55.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c72ffd700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c74101400 0x7f0c74199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:16:55.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c737fe700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c74107b20 0x7f0c7419d9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00b970 tx=0x7f0c6c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:55.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c5bfff700 1 -- 172.21.15.31:0/426270667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c019070 con 0x7f0c74107b20 2024-04-25T13:16:55.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c7950a700 1 -- 172.21.15.31:0/426270667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7419e420 con 0x7f0c74107b20 2024-04-25T13:16:55.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c5bfff700 1 -- 172.21.15.31:0/426270667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c6c014440 con 0x7f0c74107b20 2024-04-25T13:16:55.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.283+0000 7f0c5bfff700 1 -- 172.21.15.31:0/426270667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c00ce60 con 0x7f0c74107b20 2024-04-25T13:16:55.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.284+0000 7f0c7950a700 1 -- 172.21.15.31:0/426270667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c741a5d20 con 0x7f0c74107b20 2024-04-25T13:16:55.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.286+0000 7f0c7950a700 1 -- 172.21.15.31:0/426270667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c74069730 con 0x7f0c74107b20 2024-04-25T13:16:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.287+0000 7f0c5bfff700 1 -- 172.21.15.31:0/426270667 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0c6c01c070 con 0x7f0c74107b20 2024-04-25T13:16:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.287+0000 7f0c5bfff700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0c5c0a3c40 0x7f0c5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.287+0000 7f0c72ffd700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0c5c0a3c40 0x7f0c5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.287+0000 7f0c5bfff700 1 -- 172.21.15.31:0/426270667 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f0c6c0cf870 con 0x7f0c74107b20 2024-04-25T13:16:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.287+0000 7f0c72ffd700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0c5c0a3c40 0x7f0c5c0a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c64005950 tx=0x7f0c640058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.291+0000 7f0c5bfff700 1 -- 172.21.15.31:0/426270667 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c6c09c6f0 con 0x7f0c74107b20 2024-04-25T13:16:55.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:55 smithi066 ceph-mon[33137]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:16:55.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:55 smithi136 ceph-mon[33038]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:16:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.499+0000 7f0c7950a700 1 -- 172.21.15.31:0/426270667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c7402df80 con 0x7f0c74107b20 2024-04-25T13:16:55.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.500+0000 7f0c5bfff700 1 -- 172.21.15.31:0/426270667 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18419 (secure 0 0 0) 0x7f0c6c0bc070 con 0x7f0c74107b20 2024-04-25T13:16:55.501 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:16:55.503 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":84,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:16:53.053945+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:16:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 -- 172.21.15.31:0/426270667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0c5c0a3c40 msgr2=0x7f0c5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0c5c0a3c40 0x7f0c5c0a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c64005950 tx=0x7f0c640058e0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 -- 172.21.15.31:0/426270667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c74107b20 msgr2=0x7f0c7419d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c74107b20 0x7f0c7419d9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00b970 tx=0x7f0c6c00bd30 comp rx=0 tx=0).stop 2024-04-25T13:16:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 -- 172.21.15.31:0/426270667 shutdown_connections 2024-04-25T13:16:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c74107b20 0x7f0c7419d9f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c74101400 0x7f0c74199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0c5c0a3c40 0x7f0c5c0a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 --2- 172.21.15.31:0/426270667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c74101dc0 0x7f0c74199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 -- 172.21.15.31:0/426270667 >> 172.21.15.31:0/426270667 conn(0x7f0c74076c60 msgr2=0x7f0c740fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 -- 172.21.15.31:0/426270667 shutdown_connections 2024-04-25T13:16:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:55.503+0000 7f0c7950a700 1 -- 172.21.15.31:0/426270667 wait complete. 2024-04-25T13:16:56.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:56 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/426270667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:16:56.251 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:13:01.793580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:16:56.252 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T13:16:56.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:56 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/426270667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:16:56.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:56 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/426270667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:16:56.502 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:16:57.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:57 smithi031 ceph-mon[29457]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:16:57.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:57 smithi066 ceph-mon[33137]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:16:57.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:57 smithi136 ceph-mon[33038]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:16:58.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.215+0000 7fe268fde700 1 -- 172.21.15.31:0/544413037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe264108a70 msgr2=0x7fe264108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:58.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.215+0000 7fe268fde700 1 --2- 172.21.15.31:0/544413037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe264108a70 0x7fe264108e50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe254009990 tx=0x7fe254009ca0 comp rx=0 tx=0).stop 2024-04-25T13:16:58.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.215+0000 7fe268fde700 1 -- 172.21.15.31:0/544413037 shutdown_connections 2024-04-25T13:16:58.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.215+0000 7fe268fde700 1 --2- 172.21.15.31:0/544413037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe264103480 0x7fe26410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:58.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.215+0000 7fe268fde700 1 --2- 172.21.15.31:0/544413037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe264108a70 0x7fe264108e50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:58.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.215+0000 7fe268fde700 1 --2- 172.21.15.31:0/544413037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe264102ac0 0x7fe264102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:58.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.215+0000 7fe268fde700 1 -- 172.21.15.31:0/544413037 >> 172.21.15.31:0/544413037 conn(0x7fe264079bb0 msgr2=0x7fe264079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:58.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.215+0000 7fe268fde700 1 -- 172.21.15.31:0/544413037 shutdown_connections 2024-04-25T13:16:58.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.215+0000 7fe268fde700 1 -- 172.21.15.31:0/544413037 wait complete. 2024-04-25T13:16:58.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe268fde700 1 Processor -- start 2024-04-25T13:16:58.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe268fde700 1 -- start start 2024-04-25T13:16:58.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe268fde700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe264102ac0 0x7fe264198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:58.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe268fde700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe264103480 0x7fe264199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:58.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe268fde700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe264108a70 0x7fe26419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:58.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe268fde700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe264112c00 con 0x7fe264102ac0 2024-04-25T13:16:58.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe268fde700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe264112a80 con 0x7fe264108a70 2024-04-25T13:16:58.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe268fde700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe264112d80 con 0x7fe264103480 2024-04-25T13:16:58.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe26259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe264102ac0 0x7fe264198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe26259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe264102ac0 0x7fe264198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46272/0 (socket says 172.21.15.31:46272) 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe26259c700 1 -- 172.21.15.31:0/3019333443 learned_addr learned my addr 172.21.15.31:0/3019333443 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe262d9d700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe264108a70 0x7fe26419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe26259c700 1 -- 172.21.15.31:0/3019333443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe264103480 msgr2=0x7fe264199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.216+0000 7fe261d9b700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe264103480 0x7fe264199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe26259c700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe264103480 0x7fe264199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe26259c700 1 -- 172.21.15.31:0/3019333443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe264108a70 msgr2=0x7fe26419d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe26259c700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe264108a70 0x7fe26419d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe26259c700 1 -- 172.21.15.31:0/3019333443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26419e000 con 0x7fe264102ac0 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe262d9d700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe264108a70 0x7fe26419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe261d9b700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe264103480 0x7fe264199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe26259c700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe264102ac0 0x7fe264198ff0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe254000c00 tx=0x7fe2540047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe25b7fe700 1 -- 172.21.15.31:0/3019333443 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254027070 con 0x7fe264102ac0 2024-04-25T13:16:58.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe268fde700 1 -- 172.21.15.31:0/3019333443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26419e290 con 0x7fe264102ac0 2024-04-25T13:16:58.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe25b7fe700 1 -- 172.21.15.31:0/3019333443 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2540041d0 con 0x7fe264102ac0 2024-04-25T13:16:58.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe25b7fe700 1 -- 172.21.15.31:0/3019333443 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25400f460 con 0x7fe264102ac0 2024-04-25T13:16:58.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.217+0000 7fe268fde700 1 -- 172.21.15.31:0/3019333443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2641a5b70 con 0x7fe264102ac0 2024-04-25T13:16:58.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.218+0000 7fe268fde700 1 -- 172.21.15.31:0/3019333443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe264069730 con 0x7fe264102ac0 2024-04-25T13:16:58.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.219+0000 7fe25b7fe700 1 -- 172.21.15.31:0/3019333443 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe25402a070 con 0x7fe264102ac0 2024-04-25T13:16:58.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.220+0000 7fe25b7fe700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe2500a3c40 0x7fe2500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:58.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.220+0000 7fe261d9b700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe2500a3c40 0x7fe2500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:58.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.220+0000 7fe25b7fe700 1 -- 172.21.15.31:0/3019333443 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe2540d0f40 con 0x7fe264102ac0 2024-04-25T13:16:58.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.220+0000 7fe261d9b700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe2500a3c40 0x7fe2500a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe24c005fd0 tx=0x7fe24c005e20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:58.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.224+0000 7fe25b7fe700 1 -- 172.21.15.31:0/3019333443 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe25405eff0 con 0x7fe264102ac0 2024-04-25T13:16:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.430+0000 7fe268fde700 1 -- 172.21.15.31:0/3019333443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe26402df80 con 0x7fe264102ac0 2024-04-25T13:16:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.430+0000 7fe25b7fe700 1 -- 172.21.15.31:0/3019333443 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fe254013070 con 0x7fe264102ac0 2024-04-25T13:16:58.430 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T13:16:58.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 -- 172.21.15.31:0/3019333443 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe2500a3c40 msgr2=0x7fe2500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:58.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe2500a3c40 0x7fe2500a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe24c005fd0 tx=0x7fe24c005e20 comp rx=0 tx=0).stop 2024-04-25T13:16:58.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 -- 172.21.15.31:0/3019333443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe264102ac0 msgr2=0x7fe264198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:58.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe264102ac0 0x7fe264198ff0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe254000c00 tx=0x7fe2540047b0 comp rx=0 tx=0).stop 2024-04-25T13:16:58.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 -- 172.21.15.31:0/3019333443 shutdown_connections 2024-04-25T13:16:58.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe264108a70 0x7fe26419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:58.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe264102ac0 0x7fe264198ff0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:58.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe2500a3c40 0x7fe2500a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:58.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 --2- 172.21.15.31:0/3019333443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe264103480 0x7fe264199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:58.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 -- 172.21.15.31:0/3019333443 >> 172.21.15.31:0/3019333443 conn(0x7fe264079bb0 msgr2=0x7fe264101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:58.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 -- 172.21.15.31:0/3019333443 shutdown_connections 2024-04-25T13:16:58.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:16:58.432+0000 7fe268fde700 1 -- 172.21.15.31:0/3019333443 wait complete. 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.a on smithi031 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.d on smithi031 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.g on smithi031 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.j on smithi031 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.b on smithi066 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.e on smithi066 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.h on smithi066 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.k on smithi066 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.c on smithi136 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.f on smithi136 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.i on smithi136 2024-04-25T13:16:59.085 INFO:tasks.cephadm:Adding mds.l on smithi136 2024-04-25T13:16:59.086 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l' 2024-04-25T13:16:59.350 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:16:59.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:59 smithi066 ceph-mon[33137]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:16:59.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:16:59 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3019333443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:16:59.459 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:59 smithi136 ceph-mon[33038]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:16:59.460 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:16:59 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3019333443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:16:59.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:59 smithi031 ceph-mon[29457]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:16:59.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:16:59 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3019333443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:17:00.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.960+0000 7ff01f350700 1 -- 172.21.15.136:0/3411713143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff018102a90 msgr2=0x7ff018102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:00.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.960+0000 7ff01f350700 1 --2- 172.21.15.136:0/3411713143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff018102a90 0x7ff018102eb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b0a0 tx=0x7ff01000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.960+0000 7ff01f350700 1 -- 172.21.15.136:0/3411713143 shutdown_connections 2024-04-25T13:17:00.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.960+0000 7ff01f350700 1 --2- 172.21.15.136:0/3411713143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff018102a90 0x7ff018102eb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.960+0000 7ff01f350700 1 --2- 172.21.15.136:0/3411713143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018104630 0x7ff01810aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.960+0000 7ff01f350700 1 --2- 172.21.15.136:0/3411713143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff018103c90 0x7ff0181040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.960+0000 7ff01f350700 1 -- 172.21.15.136:0/3411713143 >> 172.21.15.136:0/3411713143 conn(0x7ff0180fe030 msgr2=0x7ff018100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:00.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.960+0000 7ff01f350700 1 -- 172.21.15.136:0/3411713143 shutdown_connections 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.960+0000 7ff01f350700 1 -- 172.21.15.136:0/3411713143 wait complete. 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.961+0000 7ff01f350700 1 Processor -- start 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.961+0000 7ff01f350700 1 -- start start 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.961+0000 7ff01f350700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff018102a90 0x7ff0180767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.961+0000 7ff01f350700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff018103c90 0x7ff018074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.961+0000 7ff01f350700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018104630 0x7ff018075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.961+0000 7ff01f350700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff01810e5a0 con 0x7ff018104630 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.961+0000 7ff01f350700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff01810e420 con 0x7ff018103c90 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.961+0000 7ff01f350700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff01810e720 con 0x7ff018102a90 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01d0ec700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff018102a90 0x7ff0180767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01d0ec700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff018102a90 0x7ff0180767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:35258/0 (socket says 172.21.15.136:35258) 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01d0ec700 1 -- 172.21.15.136:0/734194450 learned_addr learned my addr 172.21.15.136:0/734194450 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T13:17:00.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01c8eb700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff018103c90 0x7ff018074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:00.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01d8ed700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018104630 0x7ff018075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:00.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01d0ec700 1 -- 172.21.15.136:0/734194450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff018103c90 msgr2=0x7ff018074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:00.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01d0ec700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff018103c90 0x7ff018074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01d0ec700 1 -- 172.21.15.136:0/734194450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018104630 msgr2=0x7ff018075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:00.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01d0ec700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018104630 0x7ff018075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01d0ec700 1 -- 172.21.15.136:0/734194450 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff018075c20 con 0x7ff018102a90 2024-04-25T13:17:00.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01d0ec700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff018102a90 0x7ff0180767e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b070 tx=0x7ff010004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:00.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff00e7fc700 1 -- 172.21.15.136:0/734194450 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010021020 con 0x7ff018102a90 2024-04-25T13:17:00.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff00e7fc700 1 -- 172.21.15.136:0/734194450 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff010007b80 con 0x7ff018102a90 2024-04-25T13:17:00.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff01f350700 1 -- 172.21.15.136:0/734194450 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0181a2d80 con 0x7ff018102a90 2024-04-25T13:17:00.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.962+0000 7ff00e7fc700 1 -- 172.21.15.136:0/734194450 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100052a0 con 0x7ff018102a90 2024-04-25T13:17:00.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.963+0000 7ff01f350700 1 -- 172.21.15.136:0/734194450 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0181a3200 con 0x7ff018102a90 2024-04-25T13:17:00.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.964+0000 7ff01f350700 1 -- 172.21.15.136:0/734194450 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff018069730 con 0x7ff018102a90 2024-04-25T13:17:00.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.965+0000 7ff00e7fc700 1 -- 172.21.15.136:0/734194450 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff010024020 con 0x7ff018102a90 2024-04-25T13:17:00.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.965+0000 7ff00e7fc700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff0040a3cc0 0x7ff0040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:00.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.965+0000 7ff00e7fc700 1 -- 172.21.15.136:0/734194450 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff01000f020 con 0x7ff018102a90 2024-04-25T13:17:00.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.965+0000 7ff01c8eb700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff0040a3cc0 0x7ff0040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:00.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.966+0000 7ff01c8eb700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff0040a3cc0 0x7ff0040a6180 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff008004630 tx=0x7ff00800b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:00.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:00.970+0000 7ff00e7fc700 1 -- 172.21.15.136:0/734194450 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff010007d30 con 0x7ff018102a90 2024-04-25T13:17:01.177 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.177+0000 7ff01f350700 1 -- 172.21.15.136:0/734194450 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}) -- 0x7ff018063080 con 0x7ff0040a3cc0 2024-04-25T13:17:01.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.187+0000 7ff00e7fc700 1 -- 172.21.15.136:0/734194450 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff018063080 con 0x7ff0040a3cc0 2024-04-25T13:17:01.188 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mds.all update... 2024-04-25T13:17:01.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 -- 172.21.15.136:0/734194450 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff0040a3cc0 msgr2=0x7ff0040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:01.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff0040a3cc0 0x7ff0040a6180 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff008004630 tx=0x7ff00800b3c0 comp rx=0 tx=0).stop 2024-04-25T13:17:01.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 -- 172.21.15.136:0/734194450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff018102a90 msgr2=0x7ff0180767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:01.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff018102a90 0x7ff0180767e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b070 tx=0x7ff010004e40 comp rx=0 tx=0).stop 2024-04-25T13:17:01.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 -- 172.21.15.136:0/734194450 shutdown_connections 2024-04-25T13:17:01.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff0040a3cc0 0x7ff0040a6180 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:01.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff018102a90 0x7ff0180767e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:01.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff018104630 0x7ff018075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:01.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 --2- 172.21.15.136:0/734194450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff018103c90 0x7ff018074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:01.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 -- 172.21.15.136:0/734194450 >> 172.21.15.136:0/734194450 conn(0x7ff0180fe030 msgr2=0x7ff018107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:01.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 -- 172.21.15.136:0/734194450 shutdown_connections 2024-04-25T13:17:01.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T13:17:01.190+0000 7ff01f350700 1 -- 172.21.15.136:0/734194450 wait complete. 2024-04-25T13:17:01.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:01 smithi136 ceph-mon[33038]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:01.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:01 smithi066 ceph-mon[33137]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:01.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:01 smithi031 ceph-mon[29457]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:01.851 DEBUG:teuthology.orchestra.run.smithi031:mds.a> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T13:17:01.854 DEBUG:teuthology.orchestra.run.smithi031:mds.d> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T13:17:01.898 DEBUG:teuthology.orchestra.run.smithi031:mds.g> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T13:17:01.940 DEBUG:teuthology.orchestra.run.smithi031:mds.j> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T13:17:01.984 DEBUG:teuthology.orchestra.run.smithi066:mds.b> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T13:17:01.986 DEBUG:teuthology.orchestra.run.smithi066:mds.e> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T13:17:02.029 DEBUG:teuthology.orchestra.run.smithi066:mds.h> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T13:17:02.072 DEBUG:teuthology.orchestra.run.smithi066:mds.k> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T13:17:02.115 DEBUG:teuthology.orchestra.run.smithi136:mds.c> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T13:17:02.118 DEBUG:teuthology.orchestra.run.smithi136:mds.f> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T13:17:02.161 DEBUG:teuthology.orchestra.run.smithi136:mds.i> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T13:17:02.204 DEBUG:teuthology.orchestra.run.smithi136:mds.l> sudo journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T13:17:02.247 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T13:17:02.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:17:02.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.362+0000 7fed9d720640 1 -- 172.21.15.31:0/461127653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed98101b20 msgr2=0x7fed98101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.362+0000 7fed9d720640 1 --2- 172.21.15.31:0/461127653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed98101b20 0x7fed98101fa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fed90009ca0 tx=0x7fed9002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.362+0000 7fed9d720640 1 -- 172.21.15.31:0/461127653 shutdown_connections 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.362+0000 7fed9d720640 1 --2- 172.21.15.31:0/461127653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed98101b20 0x7fed98101fa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.362+0000 7fed9d720640 1 --2- 172.21.15.31:0/461127653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed98101180 0x7fed981015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.362+0000 7fed9d720640 1 --2- 172.21.15.31:0/461127653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed98100820 0x7fed98100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.362+0000 7fed9d720640 1 -- 172.21.15.31:0/461127653 >> 172.21.15.31:0/461127653 conn(0x7fed980fc440 msgr2=0x7fed980fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.362+0000 7fed9d720640 1 -- 172.21.15.31:0/461127653 shutdown_connections 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.362+0000 7fed9d720640 1 -- 172.21.15.31:0/461127653 wait complete. 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed9d720640 1 Processor -- start 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed9d720640 1 -- start start 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed9d720640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed98100820 0x7fed9819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed9d720640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed98101180 0x7fed9819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed9d720640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed98101b20 0x7fed981a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed9d720640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fed9810fd00 con 0x7fed98100820 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed9d720640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fed9810fb80 con 0x7fed98101b20 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed9d720640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fed9810fe80 con 0x7fed98101180 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed96ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed98100820 0x7fed9819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed96ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed98100820 0x7fed9819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47964/0 (socket says 172.21.15.31:47964) 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed96ffd640 1 -- 172.21.15.31:0/562750806 learned_addr learned my addr 172.21.15.31:0/562750806 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.363+0000 7fed967fc640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed98101180 0x7fed9819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed977fe640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed98101b20 0x7fed981a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed96ffd640 1 -- 172.21.15.31:0/562750806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed98101180 msgr2=0x7fed9819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed96ffd640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed98101180 0x7fed9819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed96ffd640 1 -- 172.21.15.31:0/562750806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed98101b20 msgr2=0x7fed981a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed96ffd640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed98101b20 0x7fed981a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed96ffd640 1 -- 172.21.15.31:0/562750806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed981a1f80 con 0x7fed98100820 2024-04-25T13:17:02.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed977fe640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed98101b20 0x7fed981a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed96ffd640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed98100820 0x7fed9819a360 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fed8000d770 tx=0x7fed8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed8ffff640 1 -- 172.21.15.31:0/562750806 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80017070 con 0x7fed98100820 2024-04-25T13:17:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed8ffff640 1 -- 172.21.15.31:0/562750806 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed800054d0 con 0x7fed98100820 2024-04-25T13:17:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed8ffff640 1 -- 172.21.15.31:0/562750806 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80004c30 con 0x7fed98100820 2024-04-25T13:17:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed9d720640 1 -- 172.21.15.31:0/562750806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed981a2270 con 0x7fed98100820 2024-04-25T13:17:02.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.364+0000 7fed9d720640 1 -- 172.21.15.31:0/562750806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed981a2770 con 0x7fed98100820 2024-04-25T13:17:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.365+0000 7fed9d720640 1 -- 172.21.15.31:0/562750806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed980691c0 con 0x7fed98100820 2024-04-25T13:17:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.366+0000 7fed8ffff640 1 -- 172.21.15.31:0/562750806 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fed80004e10 con 0x7fed98100820 2024-04-25T13:17:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.367+0000 7fed8ffff640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fed840a3bb0 0x7fed840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.367+0000 7fed967fc640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fed840a3bb0 0x7fed840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.367+0000 7fed8ffff640 1 -- 172.21.15.31:0/562750806 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fed800cfa20 con 0x7fed98100820 2024-04-25T13:17:02.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.367+0000 7fed967fc640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fed840a3bb0 0x7fed840a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fed9819b870 tx=0x7fed8800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:02.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.371+0000 7fed8ffff640 1 -- 172.21.15.31:0/562750806 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed8009c8a0 con 0x7fed98100820 2024-04-25T13:17:02.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:02 smithi066 ceph-mon[33137]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:02.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:02 smithi066 ceph-mon[33137]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-04-25T13:17:02.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:02.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:02.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:02.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:02.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:02.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:02.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:02.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:02 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:02 smithi136 ceph-mon[33038]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:02 smithi136 ceph-mon[33038]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-04-25T13:17:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:02.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:02 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:02.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:02 smithi031 ceph-mon[29457]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:02.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:02 smithi031 ceph-mon[29457]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-04-25T13:17:02.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:02.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:02.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:02.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:02.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:02.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:02.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:02.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:02 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:02.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.562+0000 7fed9d720640 1 -- 172.21.15.31:0/562750806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed98062b50 con 0x7fed98100820 2024-04-25T13:17:02.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.563+0000 7fed8ffff640 1 -- 172.21.15.31:0/562750806 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18419 (secure 0 0 0) 0x7fed8009c1a0 con 0x7fed98100820 2024-04-25T13:17:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:02.563 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":84,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:16:53.053945+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 -- 172.21.15.31:0/562750806 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fed840a3bb0 msgr2=0x7fed840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fed840a3bb0 0x7fed840a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fed9819b870 tx=0x7fed8800b390 comp rx=0 tx=0).stop 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 -- 172.21.15.31:0/562750806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed98100820 msgr2=0x7fed9819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed98100820 0x7fed9819a360 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fed8000d770 tx=0x7fed8000dc30 comp rx=0 tx=0).stop 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 -- 172.21.15.31:0/562750806 shutdown_connections 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fed840a3bb0 0x7fed840a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed98101b20 0x7fed981a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed98101180 0x7fed9819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 --2- 172.21.15.31:0/562750806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed98100820 0x7fed9819a360 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 -- 172.21.15.31:0/562750806 >> 172.21.15.31:0/562750806 conn(0x7fed980fc440 msgr2=0x7fed980fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 -- 172.21.15.31:0/562750806 shutdown_connections 2024-04-25T13:17:02.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.565+0000 7fed9d720640 1 -- 172.21.15.31:0/562750806 wait complete. 2024-04-25T13:17:02.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:13:01.793580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:17:02.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:17:02.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.681+0000 7f3bbfd36640 1 -- 172.21.15.31:0/1661365756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bb81029a0 msgr2=0x7f3bb81092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.681+0000 7f3bbfd36640 1 --2- 172.21.15.31:0/1661365756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bb81029a0 0x7f3bb81092c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009ca0 tx=0x7f3bb002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.681+0000 7f3bbfd36640 1 -- 172.21.15.31:0/1661365756 shutdown_connections 2024-04-25T13:17:02.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.681+0000 7f3bbfd36640 1 --2- 172.21.15.31:0/1661365756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bb81029a0 0x7f3bb81092c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.681+0000 7f3bbfd36640 1 --2- 172.21.15.31:0/1661365756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb8101fe0 0x7f3bb8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.681+0000 7f3bbfd36640 1 --2- 172.21.15.31:0/1661365756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8100e90 0x7f3bb81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.681+0000 7f3bbfd36640 1 -- 172.21.15.31:0/1661365756 >> 172.21.15.31:0/1661365756 conn(0x7f3bb80fc400 msgr2=0x7f3bb80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.681+0000 7f3bbfd36640 1 -- 172.21.15.31:0/1661365756 shutdown_connections 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.681+0000 7f3bbfd36640 1 -- 172.21.15.31:0/1661365756 wait complete. 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbfd36640 1 Processor -- start 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbfd36640 1 -- start start 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbfd36640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb8100e90 0x7f3bb8197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbfd36640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8101fe0 0x7f3bb8197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbfd36640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bb81029a0 0x7f3bb819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbfd36640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb810cab0 con 0x7f3bb81029a0 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbfd36640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb810c930 con 0x7f3bb8101fe0 2024-04-25T13:17:02.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbfd36640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb810cc30 con 0x7f3bb8100e90 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbe2d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bb81029a0 0x7f3bb819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbe2d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bb81029a0 0x7f3bb819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47970/0 (socket says 172.21.15.31:47970) 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbe2d3640 1 -- 172.21.15.31:0/74123438 learned_addr learned my addr 172.21.15.31:0/74123438 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbdad2640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb8100e90 0x7f3bb8197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbd2d1640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8101fe0 0x7f3bb8197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbe2d3640 1 -- 172.21.15.31:0/74123438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb8100e90 msgr2=0x7f3bb8197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbe2d3640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb8100e90 0x7f3bb8197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbe2d3640 1 -- 172.21.15.31:0/74123438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8101fe0 msgr2=0x7f3bb8197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbe2d3640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8101fe0 0x7f3bb8197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbe2d3640 1 -- 172.21.15.31:0/74123438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb819ed40 con 0x7f3bb81029a0 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbd2d1640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8101fe0 0x7f3bb8197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3bbe2d3640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bb81029a0 0x7f3bb819e6d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0033040 tx=0x7f3bb0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:02.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.682+0000 7f3baeffd640 1 -- 172.21.15.31:0/74123438 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0004040 con 0x7f3bb81029a0 2024-04-25T13:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.683+0000 7f3baeffd640 1 -- 172.21.15.31:0/74123438 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bb0004690 con 0x7f3bb81029a0 2024-04-25T13:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.683+0000 7f3bbfd36640 1 -- 172.21.15.31:0/74123438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb819efd0 con 0x7f3bb81029a0 2024-04-25T13:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.683+0000 7f3baeffd640 1 -- 172.21.15.31:0/74123438 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb002fe70 con 0x7f3bb81029a0 2024-04-25T13:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.683+0000 7f3bbfd36640 1 -- 172.21.15.31:0/74123438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb819f510 con 0x7f3bb81029a0 2024-04-25T13:17:02.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.683+0000 7f3bbfd36640 1 -- 172.21.15.31:0/74123438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb81081a0 con 0x7f3bb81029a0 2024-04-25T13:17:02.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.685+0000 7f3baeffd640 1 -- 172.21.15.31:0/74123438 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3bb002f610 con 0x7f3bb81029a0 2024-04-25T13:17:02.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.685+0000 7f3baeffd640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ba40a3bb0 0x7f3ba40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.685+0000 7f3bbdad2640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ba40a3bb0 0x7f3ba40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.685+0000 7f3baeffd640 1 -- 172.21.15.31:0/74123438 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f3bb00f18e0 con 0x7f3bb81029a0 2024-04-25T13:17:02.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.686+0000 7f3bbdad2640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ba40a3bb0 0x7f3ba40a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80058c0 tx=0x7f3ba800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:02.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.689+0000 7f3baeffd640 1 -- 172.21.15.31:0/74123438 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3bb00be760 con 0x7f3bb81029a0 2024-04-25T13:17:02.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.874+0000 7f3bbfd36640 1 -- 172.21.15.31:0/74123438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3bb8062b50 con 0x7f3bb81029a0 2024-04-25T13:17:02.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.875+0000 7f3baeffd640 1 -- 172.21.15.31:0/74123438 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f3bb00be060 con 0x7f3bb81029a0 2024-04-25T13:17:02.875 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T13:17:02.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 -- 172.21.15.31:0/74123438 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ba40a3bb0 msgr2=0x7f3ba40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ba40a3bb0 0x7f3ba40a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80058c0 tx=0x7f3ba800a2d0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 -- 172.21.15.31:0/74123438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bb81029a0 msgr2=0x7f3bb819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bb81029a0 0x7f3bb819e6d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0033040 tx=0x7f3bb0003ba0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 -- 172.21.15.31:0/74123438 shutdown_connections 2024-04-25T13:17:02.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ba40a3bb0 0x7f3ba40a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bb81029a0 0x7f3bb819e6d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3bb8101fe0 0x7f3bb8197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 --2- 172.21.15.31:0/74123438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb8100e90 0x7f3bb8197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 -- 172.21.15.31:0/74123438 >> 172.21.15.31:0/74123438 conn(0x7f3bb80fc400 msgr2=0x7f3bb8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:02.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 -- 172.21.15.31:0/74123438 shutdown_connections 2024-04-25T13:17:02.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.877+0000 7f3bbfd36640 1 -- 172.21.15.31:0/74123438 wait complete. 2024-04-25T13:17:02.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:02.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.989+0000 7fcaa3f56640 1 -- 172.21.15.31:0/732496678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca9c102040 msgr2=0x7fca9c1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.989+0000 7fcaa3f56640 1 --2- 172.21.15.31:0/732496678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca9c102040 0x7fca9c1024c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d910 tx=0x7fca8c02c380 comp rx=0 tx=0).stop 2024-04-25T13:17:02.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.989+0000 7fcaa3f56640 1 -- 172.21.15.31:0/732496678 shutdown_connections 2024-04-25T13:17:02.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.989+0000 7fcaa3f56640 1 --2- 172.21.15.31:0/732496678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca9c102a00 0x7fca9c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.989+0000 7fcaa3f56640 1 --2- 172.21.15.31:0/732496678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca9c102040 0x7fca9c1024c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.989+0000 7fcaa3f56640 1 --2- 172.21.15.31:0/732496678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca9c100e60 0x7fca9c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.989+0000 7fcaa3f56640 1 -- 172.21.15.31:0/732496678 >> 172.21.15.31:0/732496678 conn(0x7fca9c0fc440 msgr2=0x7fca9c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.989+0000 7fcaa3f56640 1 -- 172.21.15.31:0/732496678 shutdown_connections 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.989+0000 7fcaa3f56640 1 -- 172.21.15.31:0/732496678 wait complete. 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa3f56640 1 Processor -- start 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa3f56640 1 -- start start 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa3f56640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca9c100e60 0x7fca9c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa3f56640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca9c102040 0x7fca9c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa3f56640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca9c102a00 0x7fca9c1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa3f56640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c10b910 con 0x7fca9c102040 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa3f56640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c10b790 con 0x7fca9c102a00 2024-04-25T13:17:02.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa3f56640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c10ba90 con 0x7fca9c100e60 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa14f1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca9c102040 0x7fca9c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa14f1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca9c102040 0x7fca9c19a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47996/0 (socket says 172.21.15.31:47996) 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa14f1640 1 -- 172.21.15.31:0/2405537183 learned_addr learned my addr 172.21.15.31:0/2405537183 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.990+0000 7fcaa24f3640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca9c102a00 0x7fca9c1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fcaa1cf2640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca9c100e60 0x7fca9c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fcaa14f1640 1 -- 172.21.15.31:0/2405537183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca9c100e60 msgr2=0x7fca9c19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fcaa14f1640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca9c100e60 0x7fca9c19a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fcaa14f1640 1 -- 172.21.15.31:0/2405537183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca9c102a00 msgr2=0x7fca9c1a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fcaa14f1640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca9c102a00 0x7fca9c1a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fcaa14f1640 1 -- 172.21.15.31:0/2405537183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c1a1ff0 con 0x7fca9c102040 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fcaa24f3640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca9c102a00 0x7fca9c1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fcaa14f1640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca9c102040 0x7fca9c19a910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca8c02f040 tx=0x7fca8c0051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:02.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fca92ffd640 1 -- 172.21.15.31:0/2405537183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c004d20 con 0x7fca9c102040 2024-04-25T13:17:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fcaa3f56640 1 -- 172.21.15.31:0/2405537183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c1a2280 con 0x7fca9c102040 2024-04-25T13:17:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fca92ffd640 1 -- 172.21.15.31:0/2405537183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca8c0055d0 con 0x7fca9c102040 2024-04-25T13:17:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fca92ffd640 1 -- 172.21.15.31:0/2405537183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c003de0 con 0x7fca9c102040 2024-04-25T13:17:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.991+0000 7fcaa3f56640 1 -- 172.21.15.31:0/2405537183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c1a27c0 con 0x7fca9c102040 2024-04-25T13:17:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.992+0000 7fcaa3f56640 1 -- 172.21.15.31:0/2405537183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9c108280 con 0x7fca9c102040 2024-04-25T13:17:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.993+0000 7fca92ffd640 1 -- 172.21.15.31:0/2405537183 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fca8c00c050 con 0x7fca9c102040 2024-04-25T13:17:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.994+0000 7fca92ffd640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fca880a3dd0 0x7fca880a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.994+0000 7fcaa1cf2640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fca880a3dd0 0x7fca880a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.994+0000 7fca92ffd640 1 -- 172.21.15.31:0/2405537183 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fca8c0f0070 con 0x7fca9c102040 2024-04-25T13:17:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.994+0000 7fcaa1cf2640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fca880a3dd0 0x7fca880a6290 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fca98005fd0 tx=0x7fca98005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:03.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:02.997+0000 7fca92ffd640 1 -- 172.21.15.31:0/2405537183 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca8c0bcef0 con 0x7fca9c102040 2024-04-25T13:17:03.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:03 smithi136 ceph-mon[33038]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:03 smithi136 ceph-mon[33038]: Deploying daemon mds.l on smithi136 2024-04-25T13:17:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:03 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/562750806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:03 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/74123438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:17:03.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.224+0000 7fcaa3f56640 1 -- 172.21.15.31:0/2405537183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca9c02d5c0 con 0x7fca9c102040 2024-04-25T13:17:03.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.224+0000 7fca92ffd640 1 -- 172.21.15.31:0/2405537183 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fca8c0bc7f0 con 0x7fca9c102040 2024-04-25T13:17:03.224 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:03.224 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T13:17:03.225 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 1 2024-04-25T13:17:03.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 -- 172.21.15.31:0/2405537183 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fca880a3dd0 msgr2=0x7fca880a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:03.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fca880a3dd0 0x7fca880a6290 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fca98005fd0 tx=0x7fca98005850 comp rx=0 tx=0).stop 2024-04-25T13:17:03.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 -- 172.21.15.31:0/2405537183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca9c102040 msgr2=0x7fca9c19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:03.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca9c102040 0x7fca9c19a910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca8c02f040 tx=0x7fca8c0051d0 comp rx=0 tx=0).stop 2024-04-25T13:17:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 -- 172.21.15.31:0/2405537183 shutdown_connections 2024-04-25T13:17:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fca880a3dd0 0x7fca880a6290 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fca9c102a00 0x7fca9c1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca9c102040 0x7fca9c19a910 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 --2- 172.21.15.31:0/2405537183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca9c100e60 0x7fca9c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 -- 172.21.15.31:0/2405537183 >> 172.21.15.31:0/2405537183 conn(0x7fca9c0fc440 msgr2=0x7fca9c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 -- 172.21.15.31:0/2405537183 shutdown_connections 2024-04-25T13:17:03.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:03.226+0000 7fcaa3f56640 1 -- 172.21.15.31:0/2405537183 wait complete. 2024-04-25T13:17:03.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:03 smithi031 ceph-mon[29457]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:03.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:03 smithi031 ceph-mon[29457]: Deploying daemon mds.l on smithi136 2024-04-25T13:17:03.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:03 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/562750806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:03.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:03 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/74123438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:17:03.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:03 smithi066 ceph-mon[33137]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:03.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:03 smithi066 ceph-mon[33137]: Deploying daemon mds.l on smithi136 2024-04-25T13:17:03.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:03 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/562750806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:03.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:03 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/74123438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:17:04.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:04 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2405537183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:04.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:04 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2405537183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:04.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:04 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2405537183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:04.969 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 13:17:04 smithi136 systemd[1]: Started Ceph mds.l for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:05.238 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:05.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.353+0000 7ffa42dd8640 1 -- 172.21.15.31:0/4134803185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c101fc0 msgr2=0x7ffa3c1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:05.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.353+0000 7ffa42dd8640 1 --2- 172.21.15.31:0/4134803185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c101fc0 0x7ffa3c1023e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009ca0 tx=0x7ffa3402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:05.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.353+0000 7ffa42dd8640 1 -- 172.21.15.31:0/4134803185 shutdown_connections 2024-04-25T13:17:05.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.353+0000 7ffa42dd8640 1 --2- 172.21.15.31:0/4134803185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa3c103b70 0x7ffa3c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:05.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.353+0000 7ffa42dd8640 1 --2- 172.21.15.31:0/4134803185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa3c1031b0 0x7ffa3c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:05.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.353+0000 7ffa42dd8640 1 --2- 172.21.15.31:0/4134803185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c101fc0 0x7ffa3c1023e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:05.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.353+0000 7ffa42dd8640 1 -- 172.21.15.31:0/4134803185 >> 172.21.15.31:0/4134803185 conn(0x7ffa3c0fd530 msgr2=0x7ffa3c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:05.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.353+0000 7ffa42dd8640 1 -- 172.21.15.31:0/4134803185 shutdown_connections 2024-04-25T13:17:05.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.353+0000 7ffa42dd8640 1 -- 172.21.15.31:0/4134803185 wait complete. 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa42dd8640 1 Processor -- start 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa42dd8640 1 -- start start 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa42dd8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c101fc0 0x7ffa3c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa42dd8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa3c1031b0 0x7ffa3c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa40b74640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c101fc0 0x7ffa3c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa42dd8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa3c103b70 0x7ffa3c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa42dd8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c10db10 con 0x7ffa3c101fc0 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa42dd8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c10d990 con 0x7ffa3c1031b0 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa42dd8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c10dc90 con 0x7ffa3c103b70 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa40b74640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c101fc0 0x7ffa3c075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48024/0 (socket says 172.21.15.31:48024) 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa40b74640 1 -- 172.21.15.31:0/1177161667 learned_addr learned my addr 172.21.15.31:0/1177161667 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa3bfff640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa3c1031b0 0x7ffa3c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:05.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.354+0000 7ffa41375640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa3c103b70 0x7ffa3c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.355+0000 7ffa40b74640 1 -- 172.21.15.31:0/1177161667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa3c103b70 msgr2=0x7ffa3c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.355+0000 7ffa40b74640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa3c103b70 0x7ffa3c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.355+0000 7ffa40b74640 1 -- 172.21.15.31:0/1177161667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa3c1031b0 msgr2=0x7ffa3c074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.355+0000 7ffa40b74640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa3c1031b0 0x7ffa3c074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.355+0000 7ffa40b74640 1 -- 172.21.15.31:0/1177161667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c075110 con 0x7ffa3c101fc0 2024-04-25T13:17:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.355+0000 7ffa40b74640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c101fc0 0x7ffa3c075d00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffa34000f80 tx=0x7ffa34003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.355+0000 7ffa39ffb640 1 -- 172.21.15.31:0/1177161667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340317c0 con 0x7ffa3c101fc0 2024-04-25T13:17:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.355+0000 7ffa39ffb640 1 -- 172.21.15.31:0/1177161667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa340044f0 con 0x7ffa3c101fc0 2024-04-25T13:17:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.355+0000 7ffa42dd8640 1 -- 172.21.15.31:0/1177161667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c1a22d0 con 0x7ffa3c101fc0 2024-04-25T13:17:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.355+0000 7ffa39ffb640 1 -- 172.21.15.31:0/1177161667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3402fd60 con 0x7ffa3c101fc0 2024-04-25T13:17:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.355+0000 7ffa42dd8640 1 -- 172.21.15.31:0/1177161667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c1a2820 con 0x7ffa3c101fc0 2024-04-25T13:17:05.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.356+0000 7ffa42dd8640 1 -- 172.21.15.31:0/1177161667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3c109360 con 0x7ffa3c101fc0 2024-04-25T13:17:05.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.357+0000 7ffa39ffb640 1 -- 172.21.15.31:0/1177161667 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffa34004040 con 0x7ffa3c101fc0 2024-04-25T13:17:05.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.358+0000 7ffa39ffb640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ffa240a3bb0 0x7ffa240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:05.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.358+0000 7ffa39ffb640 1 -- 172.21.15.31:0/1177161667 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ffa340f14a0 con 0x7ffa3c101fc0 2024-04-25T13:17:05.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.358+0000 7ffa3bfff640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ffa240a3bb0 0x7ffa240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:05.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.359+0000 7ffa3bfff640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ffa240a3bb0 0x7ffa240a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c075a70 tx=0x7ffa3000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:05.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.363+0000 7ffa39ffb640 1 -- 172.21.15.31:0/1177161667 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa340be320 con 0x7ffa3c101fc0 2024-04-25T13:17:05.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:05 smithi136 ceph-mon[33038]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:05.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:05 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.136:6832/2316613823,v1:172.21.15.136:6833/2316613823] up:boot 2024-04-25T13:17:05.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:05 smithi136 ceph-mon[33038]: fsmap 1 up:standby 2024-04-25T13:17:05.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:05 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:17:05.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:05 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:05.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:05 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:05.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:05 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:05.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:05 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:05.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:05 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:05.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:05 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:05.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:05 smithi031 ceph-mon[29457]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:05.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:05 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.136:6832/2316613823,v1:172.21.15.136:6833/2316613823] up:boot 2024-04-25T13:17:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:05 smithi031 ceph-mon[29457]: fsmap 1 up:standby 2024-04-25T13:17:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:05 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:17:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:05 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:05 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:05 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:05 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:05 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:05 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:05.497 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:05 smithi066 ceph-mon[33137]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:05.497 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:05 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.136:6832/2316613823,v1:172.21.15.136:6833/2316613823] up:boot 2024-04-25T13:17:05.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:05 smithi066 ceph-mon[33137]: fsmap 1 up:standby 2024-04-25T13:17:05.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:05 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:17:05.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:05 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:05.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:05 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:05.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:05 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:05.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:05 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:05.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:05 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:05.498 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:05 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:05.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.591+0000 7ffa42dd8640 1 -- 172.21.15.31:0/1177161667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa3c02d5c0 con 0x7ffa3c101fc0 2024-04-25T13:17:05.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.592+0000 7ffa39ffb640 1 -- 172.21.15.31:0/1177161667 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7ffa340a8090 con 0x7ffa3c101fc0 2024-04-25T13:17:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:05.592 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:17:05.592 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-04-25T13:17:05.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 -- 172.21.15.31:0/1177161667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ffa240a3bb0 msgr2=0x7ffa240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:05.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ffa240a3bb0 0x7ffa240a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c075a70 tx=0x7ffa3000b390 comp rx=0 tx=0).stop 2024-04-25T13:17:05.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 -- 172.21.15.31:0/1177161667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c101fc0 msgr2=0x7ffa3c075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:05.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c101fc0 0x7ffa3c075d00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffa34000f80 tx=0x7ffa34003ba0 comp rx=0 tx=0).stop 2024-04-25T13:17:05.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 -- 172.21.15.31:0/1177161667 shutdown_connections 2024-04-25T13:17:05.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ffa240a3bb0 0x7ffa240a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:05.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa3c103b70 0x7ffa3c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:05.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa3c1031b0 0x7ffa3c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:05.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 --2- 172.21.15.31:0/1177161667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa3c101fc0 0x7ffa3c075d00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:05.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 -- 172.21.15.31:0/1177161667 >> 172.21.15.31:0/1177161667 conn(0x7ffa3c0fd530 msgr2=0x7ffa3c106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:05.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 -- 172.21.15.31:0/1177161667 shutdown_connections 2024-04-25T13:17:05.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:05.594+0000 7ffa42dd8640 1 -- 172.21.15.31:0/1177161667 wait complete. 2024-04-25T13:17:06.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:06 smithi136 ceph-mon[33038]: Deploying daemon mds.k on smithi066 2024-04-25T13:17:06.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:06 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1177161667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:06.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:06 smithi031 ceph-mon[29457]: Deploying daemon mds.k on smithi066 2024-04-25T13:17:06.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:06 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1177161667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:06.500 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:06 smithi066 ceph-mon[33137]: Deploying daemon mds.k on smithi066 2024-04-25T13:17:06.501 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:06 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1177161667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:07.211 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:17:07 smithi066 systemd[1]: Starting Ceph mds.k for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:17:07.467 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:07 smithi066 ceph-mon[33137]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:07.468 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:17:07 smithi066 podman[62684]: 2024-04-25 13:17:07.206660653 +0000 UTC m=+0.018650089 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:07.468 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:17:07 smithi066 podman[62684]: 2024-04-25 13:17:07.323169899 +0000 UTC m=+0.135159337 container create cb5e6fdeda399e3ff54e117dcebbe8c560d542b2a0b30c5ead968fef3e1f7da6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T13:17:07.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:07 smithi136 ceph-mon[33038]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:07.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:07 smithi031 ceph-mon[29457]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:07.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 -- 172.21.15.31:0/2892598112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5db0107d10 msgr2=0x7f5db010a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 --2- 172.21.15.31:0/2892598112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5db0107d10 0x7f5db010a100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009ca0 tx=0x7f5da802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 -- 172.21.15.31:0/2892598112 shutdown_connections 2024-04-25T13:17:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 --2- 172.21.15.31:0/2892598112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5db0107d10 0x7f5db010a100 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 --2- 172.21.15.31:0/2892598112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db01053e0 0x7f5db01077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 --2- 172.21.15.31:0/2892598112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5db0102ab0 0x7f5db0104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 -- 172.21.15.31:0/2892598112 >> 172.21.15.31:0/2892598112 conn(0x7f5db00fc400 msgr2=0x7f5db00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 -- 172.21.15.31:0/2892598112 shutdown_connections 2024-04-25T13:17:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 -- 172.21.15.31:0/2892598112 wait complete. 2024-04-25T13:17:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 Processor -- start 2024-04-25T13:17:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 -- start start 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db0102ab0 0x7f5db019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5db01053e0 0x7f5db019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5db0107d10 0x7f5db01a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5db010db40 con 0x7f5db01053e0 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5db010d9c0 con 0x7f5db0107d10 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.722+0000 7f5db6ce8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5db010dcc0 con 0x7f5db0102ab0 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5daffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5db01053e0 0x7f5db019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5db5285640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5db0107d10 0x7f5db01a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5daffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5db01053e0 0x7f5db019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48046/0 (socket says 172.21.15.31:48046) 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5daffff640 1 -- 172.21.15.31:0/1761001661 learned_addr learned my addr 172.21.15.31:0/1761001661 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5db4a84640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db0102ab0 0x7f5db019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:07.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5daffff640 1 -- 172.21.15.31:0/1761001661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db0102ab0 msgr2=0x7f5db019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5daffff640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db0102ab0 0x7f5db019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5daffff640 1 -- 172.21.15.31:0/1761001661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5db0107d10 msgr2=0x7f5db01a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5daffff640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5db0107d10 0x7f5db01a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5daffff640 1 -- 172.21.15.31:0/1761001661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db01a1fd0 con 0x7f5db01053e0 2024-04-25T13:17:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5db5285640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5db0107d10 0x7f5db01a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:17:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.723+0000 7f5daffff640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5db01053e0 0x7f5db019a8f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5da400d740 tx=0x7f5da400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.724+0000 7f5dadffb640 1 -- 172.21.15.31:0/1761001661 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4017070 con 0x7f5db01053e0 2024-04-25T13:17:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.724+0000 7f5dadffb640 1 -- 172.21.15.31:0/1761001661 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5da400de50 con 0x7f5db01053e0 2024-04-25T13:17:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.724+0000 7f5db6ce8640 1 -- 172.21.15.31:0/1761001661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db01a22c0 con 0x7f5db01053e0 2024-04-25T13:17:07.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.724+0000 7f5dadffb640 1 -- 172.21.15.31:0/1761001661 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4004ce0 con 0x7f5db01053e0 2024-04-25T13:17:07.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.724+0000 7f5db6ce8640 1 -- 172.21.15.31:0/1761001661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db01a27c0 con 0x7f5db01053e0 2024-04-25T13:17:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.725+0000 7f5db6ce8640 1 -- 172.21.15.31:0/1761001661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db0109800 con 0x7f5db01053e0 2024-04-25T13:17:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.726+0000 7f5dadffb640 1 -- 172.21.15.31:0/1761001661 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5da4004e80 con 0x7f5db01053e0 2024-04-25T13:17:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.727+0000 7f5dadffb640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5d980a3bb0 0x7f5d980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.727+0000 7f5db4a84640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5d980a3bb0 0x7f5d980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.727+0000 7f5dadffb640 1 -- 172.21.15.31:0/1761001661 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f5da40cf960 con 0x7f5db01053e0 2024-04-25T13:17:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.727+0000 7f5db4a84640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5d980a3bb0 0x7f5d980a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009b60 tx=0x7f5da0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:07.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.733+0000 7f5dadffb640 1 -- 172.21.15.31:0/1761001661 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5da409c7c0 con 0x7f5db01053e0 2024-04-25T13:17:07.908 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:17:07 smithi066 podman[62684]: 2024-04-25 13:17:07.592152497 +0000 UTC m=+0.404141935 container init cb5e6fdeda399e3ff54e117dcebbe8c560d542b2a0b30c5ead968fef3e1f7da6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:17:07.908 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:17:07 smithi066 podman[62684]: 2024-04-25 13:17:07.594814177 +0000 UTC m=+0.406803613 container start cb5e6fdeda399e3ff54e117dcebbe8c560d542b2a0b30c5ead968fef3e1f7da6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-25T13:17:07.908 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:17:07 smithi066 ceph-mds[62703]: -- 172.21.15.66:0/3690062692 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55efa3bb9a40 con 0x55efa3b8cc00 2024-04-25T13:17:07.908 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:17:07 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-k[62699]: starting mds.k at 2024-04-25T13:17:07.908 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:17:07 smithi066 bash[62684]: cb5e6fdeda399e3ff54e117dcebbe8c560d542b2a0b30c5ead968fef3e1f7da6 2024-04-25T13:17:07.908 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:17:07 smithi066 systemd[1]: Started Ceph mds.k for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:07.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.977+0000 7f5db6ce8640 1 -- 172.21.15.31:0/1761001661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5db002d770 con 0x7f5db01053e0 2024-04-25T13:17:07.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.977+0000 7f5dadffb640 1 -- 172.21.15.31:0/1761001661 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f5da409c0c0 con 0x7f5db01053e0 2024-04-25T13:17:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:07.978 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:17:07.979 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-04-25T13:17:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.980+0000 7f5db6ce8640 1 -- 172.21.15.31:0/1761001661 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5d980a3bb0 msgr2=0x7f5d980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.980+0000 7f5db6ce8640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5d980a3bb0 0x7f5d980a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009b60 tx=0x7f5da0005f90 comp rx=0 tx=0).stop 2024-04-25T13:17:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.980+0000 7f5db6ce8640 1 -- 172.21.15.31:0/1761001661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5db01053e0 msgr2=0x7f5db019a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.980+0000 7f5db6ce8640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5db01053e0 0x7f5db019a8f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5da400d740 tx=0x7f5da400dc00 comp rx=0 tx=0).stop 2024-04-25T13:17:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.980+0000 7f5db6ce8640 1 -- 172.21.15.31:0/1761001661 shutdown_connections 2024-04-25T13:17:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.980+0000 7f5db6ce8640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5d980a3bb0 0x7f5d980a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.980+0000 7f5db6ce8640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5db0107d10 0x7f5db01a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.980+0000 7f5db6ce8640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5db01053e0 0x7f5db019a8f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.980+0000 7f5db6ce8640 1 --2- 172.21.15.31:0/1761001661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db0102ab0 0x7f5db019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.980+0000 7f5db6ce8640 1 -- 172.21.15.31:0/1761001661 >> 172.21.15.31:0/1761001661 conn(0x7f5db00fc400 msgr2=0x7f5db01021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:07.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.981+0000 7f5db6ce8640 1 -- 172.21.15.31:0/1761001661 shutdown_connections 2024-04-25T13:17:07.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:07.981+0000 7f5db6ce8640 1 -- 172.21.15.31:0/1761001661 wait complete. 2024-04-25T13:17:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:08.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:08.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: Deploying daemon mds.j on smithi031 2024-04-25T13:17:08.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:08.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1761001661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:08.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.66:6832/650781414,v1:172.21.15.66:6833/650781414] up:boot 2024-04-25T13:17:08.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: fsmap 2 up:standby 2024-04-25T13:17:08.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:08 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:17:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: Deploying daemon mds.j on smithi031 2024-04-25T13:17:09.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:09.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1761001661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:09.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.66:6832/650781414,v1:172.21.15.66:6833/650781414] up:boot 2024-04-25T13:17:09.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: fsmap 2 up:standby 2024-04-25T13:17:09.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:08 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: Deploying daemon mds.j on smithi031 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1761001661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.66:6832/650781414,v1:172.21.15.66:6833/650781414] up:boot 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: fsmap 2 up:standby 2024-04-25T13:17:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:08 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:17:09.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:10.022 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:17:09 smithi031 systemd[1]: Starting Ceph mds.j for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:17:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.112+0000 7f1c1508a640 1 -- 172.21.15.31:0/1489455711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c10107c70 msgr2=0x7f1c1010a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.112+0000 7f1c1508a640 1 --2- 172.21.15.31:0/1489455711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c10107c70 0x7f1c1010a0c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009ca0 tx=0x7f1c0802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.112+0000 7f1c1508a640 1 -- 172.21.15.31:0/1489455711 shutdown_connections 2024-04-25T13:17:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.112+0000 7f1c1508a640 1 --2- 172.21.15.31:0/1489455711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c10107c70 0x7f1c1010a0c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.112+0000 7f1c1508a640 1 --2- 172.21.15.31:0/1489455711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c10105340 0x7f1c10107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.112+0000 7f1c1508a640 1 --2- 172.21.15.31:0/1489455711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c10102a10 0x7f1c10104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.112+0000 7f1c1508a640 1 -- 172.21.15.31:0/1489455711 >> 172.21.15.31:0/1489455711 conn(0x7f1c100fc400 msgr2=0x7f1c100fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.112+0000 7f1c1508a640 1 -- 172.21.15.31:0/1489455711 shutdown_connections 2024-04-25T13:17:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.112+0000 7f1c1508a640 1 -- 172.21.15.31:0/1489455711 wait complete. 2024-04-25T13:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c1508a640 1 Processor -- start 2024-04-25T13:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c1508a640 1 -- start start 2024-04-25T13:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c1508a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c10102a10 0x7f1c101981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c1508a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c10105340 0x7f1c10198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c1508a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c10107c70 0x7f1c1019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c1508a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010db30 con 0x7f1c10102a10 2024-04-25T13:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c1508a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010d9b0 con 0x7f1c10105340 2024-04-25T13:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c1508a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010dcb0 con 0x7f1c10107c70 2024-04-25T13:17:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c0ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c10102a10 0x7f1c101981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c0ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c10102a10 0x7f1c101981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48070/0 (socket says 172.21.15.31:48070) 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c0ed9d640 1 -- 172.21.15.31:0/1920229752 learned_addr learned my addr 172.21.15.31:0/1920229752 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.113+0000 7f1c07fff640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c10105340 0x7f1c10198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c0f59e640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c10107c70 0x7f1c1019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c0ed9d640 1 -- 172.21.15.31:0/1920229752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c10107c70 msgr2=0x7f1c1019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c0ed9d640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c10107c70 0x7f1c1019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c0ed9d640 1 -- 172.21.15.31:0/1920229752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c10105340 msgr2=0x7f1c10198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c0ed9d640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c10105340 0x7f1c10198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c0ed9d640 1 -- 172.21.15.31:0/1920229752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1019fe00 con 0x7f1c10102a10 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c0f59e640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c10107c70 0x7f1c1019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c07fff640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c10105340 0x7f1c10198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c0ed9d640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c10102a10 0x7f1c101981e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e9f0 tx=0x7f1c0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:10.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c0cd99640 1 -- 172.21.15.31:0/1920229752 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000cd80 con 0x7f1c10102a10 2024-04-25T13:17:10.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c1508a640 1 -- 172.21.15.31:0/1920229752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c101a00f0 con 0x7f1c10102a10 2024-04-25T13:17:10.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c0cd99640 1 -- 172.21.15.31:0/1920229752 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c000108e0 con 0x7f1c10102a10 2024-04-25T13:17:10.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c0cd99640 1 -- 172.21.15.31:0/1920229752 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00018980 con 0x7f1c10102a10 2024-04-25T13:17:10.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.114+0000 7f1c1508a640 1 -- 172.21.15.31:0/1920229752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c101a0710 con 0x7f1c10102a10 2024-04-25T13:17:10.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.115+0000 7f1c1508a640 1 -- 172.21.15.31:0/1920229752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c100fdef0 con 0x7f1c10102a10 2024-04-25T13:17:10.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.116+0000 7f1c0cd99640 1 -- 172.21.15.31:0/1920229752 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1c00010430 con 0x7f1c10102a10 2024-04-25T13:17:10.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.117+0000 7f1c0cd99640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1bfc0a3bb0 0x7f1bfc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:10.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.117+0000 7f1c07fff640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1bfc0a3bb0 0x7f1bfc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:10.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.117+0000 7f1c0cd99640 1 -- 172.21.15.31:0/1920229752 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f1c000d0430 con 0x7f1c10102a10 2024-04-25T13:17:10.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.117+0000 7f1c07fff640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1bfc0a3bb0 0x7f1bfc0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c101996f0 tx=0x7f1bf8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:10.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.121+0000 7f1c0cd99640 1 -- 172.21.15.31:0/1920229752 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c0009d210 con 0x7f1c10102a10 2024-04-25T13:17:10.336 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:17:10 smithi031 podman[69903]: 2024-04-25 13:17:10.08263149 +0000 UTC m=+0.019038401 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:10.336 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:17:10 smithi031 podman[69903]: 2024-04-25 13:17:10.223293983 +0000 UTC m=+0.159700890 container create b246bde6ed8e866649658f1b34fa02d614b813e0f4612b7fb943a1cd020ec2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-04-25T13:17:10.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.354+0000 7f1c1508a640 1 -- 172.21.15.31:0/1920229752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c1002d5c0 con 0x7f1c10102a10 2024-04-25T13:17:10.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.354+0000 7f1c0cd99640 1 -- 172.21.15.31:0/1920229752 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f1c0009cb10 con 0x7f1c10102a10 2024-04-25T13:17:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:10.354 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:17:10.355 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 3 2024-04-25T13:17:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 -- 172.21.15.31:0/1920229752 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1bfc0a3bb0 msgr2=0x7f1bfc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1bfc0a3bb0 0x7f1bfc0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c101996f0 tx=0x7f1bf8008040 comp rx=0 tx=0).stop 2024-04-25T13:17:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 -- 172.21.15.31:0/1920229752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c10102a10 msgr2=0x7f1c101981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:10.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c10102a10 0x7f1c101981e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e9f0 tx=0x7f1c0000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 -- 172.21.15.31:0/1920229752 shutdown_connections 2024-04-25T13:17:10.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1bfc0a3bb0 0x7f1bfc0a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c10107c70 0x7f1c1019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1c10105340 0x7f1c10198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 --2- 172.21.15.31:0/1920229752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c10102a10 0x7f1c101981e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 -- 172.21.15.31:0/1920229752 >> 172.21.15.31:0/1920229752 conn(0x7f1c100fc400 msgr2=0x7f1c10103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 -- 172.21.15.31:0/1920229752 shutdown_connections 2024-04-25T13:17:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:10.357+0000 7f1c1508a640 1 -- 172.21.15.31:0/1920229752 wait complete. 2024-04-25T13:17:10.734 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:17:10 smithi031 podman[69903]: 2024-04-25 13:17:10.459497508 +0000 UTC m=+0.395904408 container init b246bde6ed8e866649658f1b34fa02d614b813e0f4612b7fb943a1cd020ec2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T13:17:10.734 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:17:10 smithi031 podman[69903]: 2024-04-25 13:17:10.46237457 +0000 UTC m=+0.398781472 container start b246bde6ed8e866649658f1b34fa02d614b813e0f4612b7fb943a1cd020ec2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T13:17:10.734 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:17:10 smithi031 ceph-mds[69943]: -- 172.21.15.31:0/255409356 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f081ef3a40 con 0x55f081ec6c00 2024-04-25T13:17:10.734 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:17:10 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-j[69937]: starting mds.j at 2024-04-25T13:17:10.734 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:17:10 smithi031 bash[69903]: b246bde6ed8e866649658f1b34fa02d614b813e0f4612b7fb943a1cd020ec2de 2024-04-25T13:17:10.734 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:17:10 smithi031 systemd[1]: Started Ceph mds.j for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:10 smithi066 ceph-mon[33137]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:11.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:10 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1920229752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:11.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:10 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:11.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:10 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:11.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:10 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:11.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:10 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:11.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:10 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:11.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:10 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:10 smithi136 ceph-mon[33038]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:10 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1920229752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:10 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:10 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:11.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:10 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:11.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:10 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:11.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:10 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:11.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:10 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:11.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:10 smithi031 ceph-mon[29457]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:11.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:10 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1920229752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:11.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:11.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:11.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:11.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:11.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:11.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:10 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:12.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:11 smithi066 ceph-mon[33137]: Deploying daemon mds.d on smithi031 2024-04-25T13:17:12.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:11 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.31:6834/2096804381,v1:172.21.15.31:6835/2096804381] up:boot 2024-04-25T13:17:12.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:11 smithi066 ceph-mon[33137]: fsmap 3 up:standby 2024-04-25T13:17:12.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:11 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:17:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:11 smithi136 ceph-mon[33038]: Deploying daemon mds.d on smithi031 2024-04-25T13:17:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:11 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.31:6834/2096804381,v1:172.21.15.31:6835/2096804381] up:boot 2024-04-25T13:17:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:11 smithi136 ceph-mon[33038]: fsmap 3 up:standby 2024-04-25T13:17:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:11 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:17:12.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:11 smithi031 ceph-mon[29457]: Deploying daemon mds.d on smithi031 2024-04-25T13:17:12.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:11 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.31:6834/2096804381,v1:172.21.15.31:6835/2096804381] up:boot 2024-04-25T13:17:12.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:11 smithi031 ceph-mon[29457]: fsmap 3 up:standby 2024-04-25T13:17:12.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:11 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:17:12.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:12.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.661+0000 7f101c54f640 1 -- 172.21.15.31:0/1904012640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1014101fc0 msgr2=0x7f10141023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.661+0000 7f101c54f640 1 --2- 172.21.15.31:0/1904012640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1014101fc0 0x7f10141023e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1004009900 tx=0x7f100402a410 comp rx=0 tx=0).stop 2024-04-25T13:17:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.661+0000 7f101c54f640 1 -- 172.21.15.31:0/1904012640 shutdown_connections 2024-04-25T13:17:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.661+0000 7f101c54f640 1 --2- 172.21.15.31:0/1904012640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1014103b70 0x7f101410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.661+0000 7f101c54f640 1 --2- 172.21.15.31:0/1904012640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10141031b0 0x7f1014103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.661+0000 7f101c54f640 1 --2- 172.21.15.31:0/1904012640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1014101fc0 0x7f10141023e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.661+0000 7f101c54f640 1 -- 172.21.15.31:0/1904012640 >> 172.21.15.31:0/1904012640 conn(0x7f10140fd550 msgr2=0x7f10140ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.661+0000 7f101c54f640 1 -- 172.21.15.31:0/1904012640 shutdown_connections 2024-04-25T13:17:12.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.661+0000 7f101c54f640 1 -- 172.21.15.31:0/1904012640 wait complete. 2024-04-25T13:17:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.662+0000 7f101c54f640 1 Processor -- start 2024-04-25T13:17:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.662+0000 7f101c54f640 1 -- start start 2024-04-25T13:17:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.662+0000 7f101c54f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1014101fc0 0x7f1014198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.662+0000 7f101c54f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10141031b0 0x7f10141987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.662+0000 7f101c54f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1014103b70 0x7f101419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.662+0000 7f101c54f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f101410db00 con 0x7f1014103b70 2024-04-25T13:17:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.662+0000 7f101c54f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f101410d980 con 0x7f10141031b0 2024-04-25T13:17:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101c54f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f101410dc80 con 0x7f1014101fc0 2024-04-25T13:17:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101aaec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1014103b70 0x7f101419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101aaec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1014103b70 0x7f101419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44260/0 (socket says 172.21.15.31:44260) 2024-04-25T13:17:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101aaec640 1 -- 172.21.15.31:0/470604991 learned_addr learned my addr 172.21.15.31:0/470604991 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:12.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101a2eb640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1014101fc0 0x7f1014198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f1019aea640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10141031b0 0x7f10141987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101aaec640 1 -- 172.21.15.31:0/470604991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1014101fc0 msgr2=0x7f1014198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101aaec640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1014101fc0 0x7f1014198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101aaec640 1 -- 172.21.15.31:0/470604991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10141031b0 msgr2=0x7f10141987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101aaec640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10141031b0 0x7f10141987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101aaec640 1 -- 172.21.15.31:0/470604991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101419fe80 con 0x7f1014103b70 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f1019aea640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10141031b0 0x7f10141987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101aaec640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1014103b70 0x7f101419f810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f100c00cad0 tx=0x7f100c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f100b7fe640 1 -- 172.21.15.31:0/470604991 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c00ae20 con 0x7f1014103b70 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f100b7fe640 1 -- 172.21.15.31:0/470604991 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f100c005020 con 0x7f1014103b70 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101c54f640 1 -- 172.21.15.31:0/470604991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10141a0170 con 0x7f1014103b70 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f100b7fe640 1 -- 172.21.15.31:0/470604991 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c00f6b0 con 0x7f1014103b70 2024-04-25T13:17:12.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.663+0000 7f101c54f640 1 -- 172.21.15.31:0/470604991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10141a06c0 con 0x7f1014103b70 2024-04-25T13:17:12.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.664+0000 7f101c54f640 1 -- 172.21.15.31:0/470604991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1014109400 con 0x7f1014103b70 2024-04-25T13:17:12.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.665+0000 7f100b7fe640 1 -- 172.21.15.31:0/470604991 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f100c0051d0 con 0x7f1014103b70 2024-04-25T13:17:12.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.666+0000 7f100b7fe640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f10000a3c30 0x7f10000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:12.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.666+0000 7f101a2eb640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f10000a3c30 0x7f10000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:12.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.667+0000 7f100b7fe640 1 -- 172.21.15.31:0/470604991 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f100c0cffa0 con 0x7f1014103b70 2024-04-25T13:17:12.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.667+0000 7f101a2eb640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f10000a3c30 0x7f10000a60f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1004009230 tx=0x7f10040314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:12.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.671+0000 7f100b7fe640 1 -- 172.21.15.31:0/470604991 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f100c014030 con 0x7f1014103b70 2024-04-25T13:17:12.863 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:17:12 smithi031 systemd[1]: Starting Ceph mds.d for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:17:12.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.909+0000 7f101c54f640 1 -- 172.21.15.31:0/470604991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1014062b50 con 0x7f1014103b70 2024-04-25T13:17:12.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.910+0000 7f100b7fe640 1 -- 172.21.15.31:0/470604991 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f100c09ce20 con 0x7f1014103b70 2024-04-25T13:17:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:12.910 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:17:12.910 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 4 2024-04-25T13:17:12.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 -- 172.21.15.31:0/470604991 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f10000a3c30 msgr2=0x7f10000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:12.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f10000a3c30 0x7f10000a60f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1004009230 tx=0x7f10040314a0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 -- 172.21.15.31:0/470604991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1014103b70 msgr2=0x7f101419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:12.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1014103b70 0x7f101419f810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f100c00cad0 tx=0x7f100c00cf90 comp rx=0 tx=0).stop 2024-04-25T13:17:12.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 -- 172.21.15.31:0/470604991 shutdown_connections 2024-04-25T13:17:12.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f10000a3c30 0x7f10000a60f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1014103b70 0x7f101419f810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10141031b0 0x7f10141987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 --2- 172.21.15.31:0/470604991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1014101fc0 0x7f1014198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 -- 172.21.15.31:0/470604991 >> 172.21.15.31:0/470604991 conn(0x7f10140fd550 msgr2=0x7f10141065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:12.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 -- 172.21.15.31:0/470604991 shutdown_connections 2024-04-25T13:17:12.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:12.912+0000 7f101c54f640 1 -- 172.21.15.31:0/470604991 wait complete. 2024-04-25T13:17:13.158 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:12 smithi031 ceph-mon[29457]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:13.158 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:17:13 smithi031 podman[70271]: 2024-04-25 13:17:12.923201182 +0000 UTC m=+0.019578797 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:13.158 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:17:13 smithi031 podman[70271]: 2024-04-25 13:17:13.044740588 +0000 UTC m=+0.141118252 container create 930113d5b4b64cd0d3398af044cf7e09090820dc4c210e335d8b7834fc55eb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:17:13.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:12 smithi066 ceph-mon[33137]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:13.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:12 smithi136 ceph-mon[33038]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:13.483 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:17:13 smithi031 podman[70271]: 2024-04-25 13:17:13.272125196 +0000 UTC m=+0.368502794 container init 930113d5b4b64cd0d3398af044cf7e09090820dc4c210e335d8b7834fc55eb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:17:13.484 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:17:13 smithi031 podman[70271]: 2024-04-25 13:17:13.274944172 +0000 UTC m=+0.371321773 container start 930113d5b4b64cd0d3398af044cf7e09090820dc4c210e335d8b7834fc55eb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-d, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T13:17:13.484 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:17:13 smithi031 ceph-mds[70293]: -- 172.21.15.31:0/4016134077 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559d2e85a40 con 0x5559d2e58800 2024-04-25T13:17:13.484 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:17:13 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-d[70289]: starting mds.d at 2024-04-25T13:17:13.484 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:17:13 smithi031 bash[70271]: 930113d5b4b64cd0d3398af044cf7e09090820dc4c210e335d8b7834fc55eb60 2024-04-25T13:17:13.484 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:17:13 smithi031 systemd[1]: Started Ceph mds.d for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:14.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:13 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/470604991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:14.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:13 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:14.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:13 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:14.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:13 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:14.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:13 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:14.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:13 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:14.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:13 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:14.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:13 smithi066 ceph-mon[33137]: Deploying daemon mds.e on smithi066 2024-04-25T13:17:14.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:13 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/470604991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:14.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:13 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:14.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:13 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:14.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:13 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:14.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:13 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:14.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:13 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:14.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:13 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:14.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:13 smithi136 ceph-mon[33038]: Deploying daemon mds.e on smithi066 2024-04-25T13:17:14.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:13 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/470604991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:14.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:13 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:14.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:13 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:14.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:13 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:14.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:13 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:14.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:13 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:14.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:13 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:14.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:13 smithi031 ceph-mon[29457]: Deploying daemon mds.e on smithi066 2024-04-25T13:17:14.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.044+0000 7f013f469640 1 -- 172.21.15.31:0/94169242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01381029a0 msgr2=0x7f01381092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.044+0000 7f013f469640 1 --2- 172.21.15.31:0/94169242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01381029a0 0x7f01381092c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0130009ca0 tx=0x7f013002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.044+0000 7f013f469640 1 -- 172.21.15.31:0/94169242 shutdown_connections 2024-04-25T13:17:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.044+0000 7f013f469640 1 --2- 172.21.15.31:0/94169242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01381029a0 0x7f01381092c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.044+0000 7f013f469640 1 --2- 172.21.15.31:0/94169242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0138101fe0 0x7f0138102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.044+0000 7f013f469640 1 --2- 172.21.15.31:0/94169242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0138100e90 0x7f01381012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.044+0000 7f013f469640 1 -- 172.21.15.31:0/94169242 >> 172.21.15.31:0/94169242 conn(0x7f01380fc400 msgr2=0x7f01380fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.044+0000 7f013f469640 1 -- 172.21.15.31:0/94169242 shutdown_connections 2024-04-25T13:17:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.044+0000 7f013f469640 1 -- 172.21.15.31:0/94169242 wait complete. 2024-04-25T13:17:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.044+0000 7f013f469640 1 Processor -- start 2024-04-25T13:17:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.044+0000 7f013f469640 1 -- start start 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013f469640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0138100e90 0x7f01381980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013f469640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0138101fe0 0x7f0138198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013f469640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01381029a0 0x7f013819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013f469640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f013810ca20 con 0x7f01381029a0 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013f469640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f013810c8a0 con 0x7f0138100e90 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013f469640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f013810cba0 con 0x7f0138101fe0 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013da06640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01381029a0 0x7f013819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013d205640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0138100e90 0x7f01381980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013d205640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0138100e90 0x7f01381980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.31:54216/0 (socket says 172.21.15.31:54216) 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013da06640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01381029a0 0x7f013819f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44310/0 (socket says 172.21.15.31:44310) 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013d205640 1 -- 172.21.15.31:0/3397030607 learned_addr learned my addr 172.21.15.31:0/3397030607 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013ca04640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0138101fe0 0x7f0138198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013d205640 1 -- 172.21.15.31:0/3397030607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0138101fe0 msgr2=0x7f0138198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013d205640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0138101fe0 0x7f0138198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013d205640 1 -- 172.21.15.31:0/3397030607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01381029a0 msgr2=0x7f013819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013d205640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01381029a0 0x7f013819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013d205640 1 -- 172.21.15.31:0/3397030607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013819fce0 con 0x7f0138100e90 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013da06640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01381029a0 0x7f013819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013d205640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0138100e90 0x7f01381980c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f012800c890 tx=0x7f012800cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f012e7fc640 1 -- 172.21.15.31:0/3397030607 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128004220 con 0x7f0138100e90 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f013f469640 1 -- 172.21.15.31:0/3397030607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013819ffd0 con 0x7f0138100e90 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.045+0000 7f012e7fc640 1 -- 172.21.15.31:0/3397030607 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01280043c0 con 0x7f0138100e90 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.046+0000 7f012e7fc640 1 -- 172.21.15.31:0/3397030607 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128004a00 con 0x7f0138100e90 2024-04-25T13:17:15.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.046+0000 7f013f469640 1 -- 172.21.15.31:0/3397030607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01381a0430 con 0x7f0138100e90 2024-04-25T13:17:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.046+0000 7f013f469640 1 -- 172.21.15.31:0/3397030607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01381a06f0 con 0x7f0138100e90 2024-04-25T13:17:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.048+0000 7f012e7fc640 1 -- 172.21.15.31:0/3397030607 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f01280079b0 con 0x7f0138100e90 2024-04-25T13:17:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.048+0000 7f012e7fc640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f01240a3bb0 0x7f01240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.048+0000 7f013ca04640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f01240a3bb0 0x7f01240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.049+0000 7f012e7fc640 1 -- 172.21.15.31:0/3397030607 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f01280d00f0 con 0x7f0138100e90 2024-04-25T13:17:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.049+0000 7f013ca04640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f01240a3bb0 0x7f01240a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f01340058c0 tx=0x7f0134005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:15.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.052+0000 7f012e7fc640 1 -- 172.21.15.31:0/3397030607 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0128014030 con 0x7f0138100e90 2024-04-25T13:17:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:14 smithi066 ceph-mon[33137]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:14 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.31:6836/1972203542,v1:172.21.15.31:6837/1972203542] up:boot 2024-04-25T13:17:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:14 smithi066 ceph-mon[33137]: fsmap 4 up:standby 2024-04-25T13:17:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:14 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:17:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:14 smithi136 ceph-mon[33038]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:14 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.31:6836/1972203542,v1:172.21.15.31:6837/1972203542] up:boot 2024-04-25T13:17:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:14 smithi136 ceph-mon[33038]: fsmap 4 up:standby 2024-04-25T13:17:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:14 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:17:15.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:14 smithi031 ceph-mon[29457]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:15.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:14 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.31:6836/1972203542,v1:172.21.15.31:6837/1972203542] up:boot 2024-04-25T13:17:15.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:14 smithi031 ceph-mon[29457]: fsmap 4 up:standby 2024-04-25T13:17:15.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:14 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:17:15.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.280+0000 7f013f469640 1 -- 172.21.15.31:0/3397030607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01381a08c0 con 0x7f0138100e90 2024-04-25T13:17:15.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.280+0000 7f012e7fc640 1 -- 172.21.15.31:0/3397030607 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7f012809ced0 con 0x7f0138100e90 2024-04-25T13:17:15.280 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:15.281 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:17:15.281 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 5 2024-04-25T13:17:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 -- 172.21.15.31:0/3397030607 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f01240a3bb0 msgr2=0x7f01240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f01240a3bb0 0x7f01240a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f01340058c0 tx=0x7f0134005850 comp rx=0 tx=0).stop 2024-04-25T13:17:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 -- 172.21.15.31:0/3397030607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0138100e90 msgr2=0x7f01381980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0138100e90 0x7f01381980c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f012800c890 tx=0x7f012800cd50 comp rx=0 tx=0).stop 2024-04-25T13:17:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 -- 172.21.15.31:0/3397030607 shutdown_connections 2024-04-25T13:17:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f01240a3bb0 0x7f01240a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01381029a0 0x7f013819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0138101fe0 0x7f0138198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 --2- 172.21.15.31:0/3397030607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0138100e90 0x7f01381980c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 -- 172.21.15.31:0/3397030607 >> 172.21.15.31:0/3397030607 conn(0x7f01380fc400 msgr2=0x7f01380fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 -- 172.21.15.31:0/3397030607 shutdown_connections 2024-04-25T13:17:15.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:15.283+0000 7f013f469640 1 -- 172.21.15.31:0/3397030607 wait complete. 2024-04-25T13:17:15.879 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:17:15 smithi066 systemd[1]: Starting Ceph mds.e for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:17:16.135 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:15 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3397030607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:16.136 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:17:16 smithi066 podman[63002]: 2024-04-25 13:17:15.904823991 +0000 UTC m=+0.018708916 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:16.136 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:17:16 smithi066 podman[63002]: 2024-04-25 13:17:16.003238114 +0000 UTC m=+0.117123011 container create 549fb8454996fa3fb1da6bf28b3b9030d3577aee7f88ddb3963b428b467a9027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T13:17:16.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:15 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3397030607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:16.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:15 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3397030607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:16.658 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:17:16 smithi066 podman[63002]: 2024-04-25 13:17:16.272560874 +0000 UTC m=+0.386445774 container init 549fb8454996fa3fb1da6bf28b3b9030d3577aee7f88ddb3963b428b467a9027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:17:16.658 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:17:16 smithi066 podman[63002]: 2024-04-25 13:17:16.275246371 +0000 UTC m=+0.389131278 container start 549fb8454996fa3fb1da6bf28b3b9030d3577aee7f88ddb3963b428b467a9027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T13:17:16.658 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:17:16 smithi066 ceph-mds[63022]: -- 172.21.15.66:0/95545682 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557cf379ba40 con 0x557cf376f000 2024-04-25T13:17:16.658 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:17:16 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-e[63018]: starting mds.e at 2024-04-25T13:17:16.659 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:17:16 smithi066 bash[63002]: 549fb8454996fa3fb1da6bf28b3b9030d3577aee7f88ddb3963b428b467a9027 2024-04-25T13:17:16.659 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:17:16 smithi066 systemd[1]: Started Ceph mds.e for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:17.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.410+0000 7f1f92089640 1 -- 172.21.15.31:0/2892336459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f8c101b00 msgr2=0x7f1f8c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.410+0000 7f1f92089640 1 --2- 172.21.15.31:0/2892336459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f8c101b00 0x7f1f8c101f80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009ca0 tx=0x7f1f8402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.410+0000 7f1f92089640 1 -- 172.21.15.31:0/2892336459 shutdown_connections 2024-04-25T13:17:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.410+0000 7f1f92089640 1 --2- 172.21.15.31:0/2892336459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f8c101b00 0x7f1f8c101f80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.410+0000 7f1f92089640 1 --2- 172.21.15.31:0/2892336459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8c101160 0x7f1f8c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.410+0000 7f1f92089640 1 --2- 172.21.15.31:0/2892336459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c100800 0x7f1f8c100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.410+0000 7f1f92089640 1 -- 172.21.15.31:0/2892336459 >> 172.21.15.31:0/2892336459 conn(0x7f1f8c0fc400 msgr2=0x7f1f8c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.410+0000 7f1f92089640 1 -- 172.21.15.31:0/2892336459 shutdown_connections 2024-04-25T13:17:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.410+0000 7f1f92089640 1 -- 172.21.15.31:0/2892336459 wait complete. 2024-04-25T13:17:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.410+0000 7f1f92089640 1 Processor -- start 2024-04-25T13:17:17.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f92089640 1 -- start start 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f92089640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f8c100800 0x7f1f8c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f92089640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c101160 0x7f1f8c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f92089640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8c101b00 0x7f1f8c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f92089640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c10fce0 con 0x7f1f8c100800 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f92089640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c10fb60 con 0x7f1f8c101b00 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f92089640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c10fe60 con 0x7f1f8c101160 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f8b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f8c100800 0x7f1f8c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f8b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f8c100800 0x7f1f8c19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44320/0 (socket says 172.21.15.31:44320) 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f8b7fe640 1 -- 172.21.15.31:0/4100515566 learned_addr learned my addr 172.21.15.31:0/4100515566 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f8affd640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c101160 0x7f1f8c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f8bfff640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8c101b00 0x7f1f8c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f8b7fe640 1 -- 172.21.15.31:0/4100515566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c101160 msgr2=0x7f1f8c19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f8b7fe640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c101160 0x7f1f8c19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f8b7fe640 1 -- 172.21.15.31:0/4100515566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8c101b00 msgr2=0x7f1f8c1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f8b7fe640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8c101b00 0x7f1f8c1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:17.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.411+0000 7f1f8b7fe640 1 -- 172.21.15.31:0/4100515566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c1a1f70 con 0x7f1f8c100800 2024-04-25T13:17:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.412+0000 7f1f8b7fe640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f8c100800 0x7f1f8c19a350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00e9f0 tx=0x7f1f7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.412+0000 7f1f88ff9640 1 -- 172.21.15.31:0/4100515566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00cd80 con 0x7f1f8c100800 2024-04-25T13:17:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.412+0000 7f1f88ff9640 1 -- 172.21.15.31:0/4100515566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7c0054d0 con 0x7f1f8c100800 2024-04-25T13:17:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.412+0000 7f1f92089640 1 -- 172.21.15.31:0/4100515566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c1a2260 con 0x7f1f8c100800 2024-04-25T13:17:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.412+0000 7f1f88ff9640 1 -- 172.21.15.31:0/4100515566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c010690 con 0x7f1f8c100800 2024-04-25T13:17:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.412+0000 7f1f92089640 1 -- 172.21.15.31:0/4100515566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c1a2760 con 0x7f1f8c100800 2024-04-25T13:17:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.413+0000 7f1f92089640 1 -- 172.21.15.31:0/4100515566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8c10b6d0 con 0x7f1f8c100800 2024-04-25T13:17:17.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.415+0000 7f1f88ff9640 1 -- 172.21.15.31:0/4100515566 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1f7c004e30 con 0x7f1f8c100800 2024-04-25T13:17:17.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.415+0000 7f1f88ff9640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f740a3bb0 0x7f1f740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:17.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.416+0000 7f1f8affd640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f740a3bb0 0x7f1f740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:17.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.416+0000 7f1f88ff9640 1 -- 172.21.15.31:0/4100515566 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f1f7c0d0610 con 0x7f1f8c100800 2024-04-25T13:17:17.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.416+0000 7f1f8affd640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f740a3bb0 0x7f1f740a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c19b860 tx=0x7f1f80005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:17.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.419+0000 7f1f88ff9640 1 -- 172.21.15.31:0/4100515566 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f7c053ff0 con 0x7f1f8c100800 2024-04-25T13:17:17.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.645+0000 7f1f92089640 1 -- 172.21.15.31:0/4100515566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f8c02d5c0 con 0x7f1f8c100800 2024-04-25T13:17:17.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.645+0000 7f1f88ff9640 1 -- 172.21.15.31:0/4100515566 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f1f7c09d3f0 con 0x7f1f8c100800 2024-04-25T13:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:17.645 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T13:17:17.645 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-04-25T13:17:17.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 -- 172.21.15.31:0/4100515566 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f740a3bb0 msgr2=0x7f1f740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:17.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f740a3bb0 0x7f1f740a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c19b860 tx=0x7f1f80005f90 comp rx=0 tx=0).stop 2024-04-25T13:17:17.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 -- 172.21.15.31:0/4100515566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f8c100800 msgr2=0x7f1f8c19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:17.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f8c100800 0x7f1f8c19a350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00e9f0 tx=0x7f1f7c00edf0 comp rx=0 tx=0).stop 2024-04-25T13:17:17.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 -- 172.21.15.31:0/4100515566 shutdown_connections 2024-04-25T13:17:17.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f740a3bb0 0x7f1f740a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:17.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f8c101b00 0x7f1f8c1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:17.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f8c101160 0x7f1f8c19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:17.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 --2- 172.21.15.31:0/4100515566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f8c100800 0x7f1f8c19a350 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:17.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 -- 172.21.15.31:0/4100515566 >> 172.21.15.31:0/4100515566 conn(0x7f1f8c0fc400 msgr2=0x7f1f8c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:17.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 -- 172.21.15.31:0/4100515566 shutdown_connections 2024-04-25T13:17:17.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:17.647+0000 7f1f92089640 1 -- 172.21.15.31:0/4100515566 wait complete. 2024-04-25T13:17:17.657 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:17 smithi031 ceph-mon[29457]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:17.657 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:17.657 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:17.657 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:17.657 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:17.657 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:17.657 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:17.658 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:17 smithi031 ceph-mon[29457]: Deploying daemon mds.f on smithi136 2024-04-25T13:17:17.658 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:17 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.66:6834/2836275175,v1:172.21.15.66:6835/2836275175] up:boot 2024-04-25T13:17:17.658 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:17 smithi031 ceph-mon[29457]: fsmap 5 up:standby 2024-04-25T13:17:17.658 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:17 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:17:17.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:17 smithi066 ceph-mon[33137]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:17.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:17.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:17.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:17.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:17.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:17.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:17.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:17 smithi066 ceph-mon[33137]: Deploying daemon mds.f on smithi136 2024-04-25T13:17:17.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:17 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.66:6834/2836275175,v1:172.21.15.66:6835/2836275175] up:boot 2024-04-25T13:17:17.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:17 smithi066 ceph-mon[33137]: fsmap 5 up:standby 2024-04-25T13:17:17.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:17 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:17:17.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:17 smithi136 ceph-mon[33038]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:17.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:17.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:17.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:17.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:17.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:17.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:17.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:17 smithi136 ceph-mon[33038]: Deploying daemon mds.f on smithi136 2024-04-25T13:17:17.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:17 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.66:6834/2836275175,v1:172.21.15.66:6835/2836275175] up:boot 2024-04-25T13:17:17.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:17 smithi136 ceph-mon[33038]: fsmap 5 up:standby 2024-04-25T13:17:17.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:17 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:17:18.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:18 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/4100515566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:18.719 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:17:18 smithi136 podman[63012]: 2024-04-25 13:17:18.547181083 +0000 UTC m=+0.017907130 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:18.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:18 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/4100515566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:18.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:18 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/4100515566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:19.219 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:17:18 smithi136 podman[63012]: 2024-04-25 13:17:18.845531885 +0000 UTC m=+0.316257903 container init 2e9a08925b534a137041c6dba50e498b5f90288be736428dae62262f5cd87b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T13:17:19.219 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:17:18 smithi136 podman[63012]: 2024-04-25 13:17:18.848300668 +0000 UTC m=+0.319026689 container start 2e9a08925b534a137041c6dba50e498b5f90288be736428dae62262f5cd87b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T13:17:19.219 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:17:18 smithi136 ceph-mds[63031]: -- 172.21.15.136:0/3215809171 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56138e697a40 con 0x56138e66b000 2024-04-25T13:17:19.219 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:17:18 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-f[63027]: starting mds.f at 2024-04-25T13:17:19.220 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:17:18 smithi136 bash[63012]: 2e9a08925b534a137041c6dba50e498b5f90288be736428dae62262f5cd87b91 2024-04-25T13:17:19.220 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:17:18 smithi136 systemd[1]: Started Ceph mds.f for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:19.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:19.687 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:19 smithi031 ceph-mon[29457]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:19.687 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:19.687 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:19.687 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:19.687 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:19.687 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:19.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:19 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:19.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:19 smithi136 ceph-mon[33038]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:19.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:19.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:19.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:19.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:19.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:19.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:19 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:19.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.777+0000 7f6c847fe640 1 -- 172.21.15.31:0/3521950709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c7c100e80 msgr2=0x7f6c7c1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:19.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.777+0000 7f6c847fe640 1 --2- 172.21.15.31:0/3521950709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c7c100e80 0x7f6c7c1012a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009900 tx=0x7f6c6c02a410 comp rx=0 tx=0).stop 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.777+0000 7f6c847fe640 1 -- 172.21.15.31:0/3521950709 shutdown_connections 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.777+0000 7f6c847fe640 1 --2- 172.21.15.31:0/3521950709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c7c102a20 0x7f6c7c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.777+0000 7f6c847fe640 1 --2- 172.21.15.31:0/3521950709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c7c102060 0x7f6c7c1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.777+0000 7f6c847fe640 1 --2- 172.21.15.31:0/3521950709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c7c100e80 0x7f6c7c1012a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.777+0000 7f6c847fe640 1 -- 172.21.15.31:0/3521950709 >> 172.21.15.31:0/3521950709 conn(0x7f6c7c0fc440 msgr2=0x7f6c7c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.777+0000 7f6c847fe640 1 -- 172.21.15.31:0/3521950709 shutdown_connections 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.777+0000 7f6c847fe640 1 -- 172.21.15.31:0/3521950709 wait complete. 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.777+0000 7f6c847fe640 1 Processor -- start 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.777+0000 7f6c847fe640 1 -- start start 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c847fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c7c100e80 0x7f6c7c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c847fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c7c102060 0x7f6c7c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c847fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c7c102a20 0x7f6c7c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:19.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c847fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c10ca20 con 0x7f6c7c102060 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c847fe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c10c8a0 con 0x7f6c7c100e80 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c847fe640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c10cba0 con 0x7f6c7c102a20 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c81d99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c7c102060 0x7f6c7c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c81d99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c7c102060 0x7f6c7c1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44334/0 (socket says 172.21.15.31:44334) 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c81d99640 1 -- 172.21.15.31:0/2542895817 learned_addr learned my addr 172.21.15.31:0/2542895817 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c8259a640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c7c100e80 0x7f6c7c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c82d9b640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c7c102a20 0x7f6c7c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c81d99640 1 -- 172.21.15.31:0/2542895817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c7c102a20 msgr2=0x7f6c7c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c81d99640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c7c102a20 0x7f6c7c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c81d99640 1 -- 172.21.15.31:0/2542895817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c7c100e80 msgr2=0x7f6c7c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c81d99640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c7c100e80 0x7f6c7c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c81d99640 1 -- 172.21.15.31:0/2542895817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c19fe70 con 0x7f6c7c102060 2024-04-25T13:17:19.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c81d99640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c7c102060 0x7f6c7c1987e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800d770 tx=0x7f6c7800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:19.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c737fe640 1 -- 172.21.15.31:0/2542895817 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78017070 con 0x7f6c7c102060 2024-04-25T13:17:19.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c737fe640 1 -- 172.21.15.31:0/2542895817 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c780054d0 con 0x7f6c7c102060 2024-04-25T13:17:19.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c737fe640 1 -- 172.21.15.31:0/2542895817 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78004c30 con 0x7f6c7c102060 2024-04-25T13:17:19.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c847fe640 1 -- 172.21.15.31:0/2542895817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c1a0160 con 0x7f6c7c102060 2024-04-25T13:17:19.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.778+0000 7f6c847fe640 1 -- 172.21.15.31:0/2542895817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c1a0660 con 0x7f6c7c102060 2024-04-25T13:17:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.779+0000 7f6c847fe640 1 -- 172.21.15.31:0/2542895817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7c1a08c0 con 0x7f6c7c102060 2024-04-25T13:17:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.781+0000 7f6c737fe640 1 -- 172.21.15.31:0/2542895817 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6c78004dd0 con 0x7f6c7c102060 2024-04-25T13:17:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.782+0000 7f6c737fe640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6c680a3c30 0x7f6c680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.782+0000 7f6c8259a640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6c680a3c30 0x7f6c680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.782+0000 7f6c737fe640 1 -- 172.21.15.31:0/2542895817 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f6c780cfd20 con 0x7f6c7c102060 2024-04-25T13:17:19.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.782+0000 7f6c8259a640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6c680a3c30 0x7f6c680a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009230 tx=0x7f6c6c0314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:19.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:19.786+0000 7f6c737fe640 1 -- 172.21.15.31:0/2542895817 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c7809c5d0 con 0x7f6c7c102060 2024-04-25T13:17:19.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:19 smithi066 ceph-mon[33137]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:19.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:19.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:19.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:19.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:19.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:19.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:19 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.012+0000 7f6c847fe640 1 -- 172.21.15.31:0/2542895817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c7c1a0a90 con 0x7f6c7c102060 2024-04-25T13:17:20.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.012+0000 7f6c737fe640 1 -- 172.21.15.31:0/2542895817 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f6c780a08d0 con 0x7f6c7c102060 2024-04-25T13:17:20.012 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:20.013 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T13:17:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.014+0000 7f6c847fe640 1 -- 172.21.15.31:0/2542895817 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6c680a3c30 msgr2=0x7f6c680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.014+0000 7f6c847fe640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6c680a3c30 0x7f6c680a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009230 tx=0x7f6c6c0314a0 comp rx=0 tx=0).stop 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.014+0000 7f6c847fe640 1 -- 172.21.15.31:0/2542895817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c7c102060 msgr2=0x7f6c7c1987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.014+0000 7f6c847fe640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c7c102060 0x7f6c7c1987e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800d770 tx=0x7f6c7800dc30 comp rx=0 tx=0).stop 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.014+0000 7f6c847fe640 1 -- 172.21.15.31:0/2542895817 shutdown_connections 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.014+0000 7f6c847fe640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6c680a3c30 0x7f6c680a60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.014+0000 7f6c847fe640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c7c102a20 0x7f6c7c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.014+0000 7f6c847fe640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c7c102060 0x7f6c7c1987e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.014+0000 7f6c847fe640 1 --2- 172.21.15.31:0/2542895817 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c7c100e80 0x7f6c7c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.014+0000 7f6c847fe640 1 -- 172.21.15.31:0/2542895817 >> 172.21.15.31:0/2542895817 conn(0x7f6c7c0fc440 msgr2=0x7f6c7c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.015+0000 7f6c847fe640 1 -- 172.21.15.31:0/2542895817 shutdown_connections 2024-04-25T13:17:20.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:20.015+0000 7f6c847fe640 1 -- 172.21.15.31:0/2542895817 wait complete. 2024-04-25T13:17:20.703 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:20 smithi136 ceph-mon[33038]: Deploying daemon mds.c on smithi136 2024-04-25T13:17:20.703 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:20 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.136:6834/3086187130,v1:172.21.15.136:6835/3086187130] up:boot 2024-04-25T13:17:20.703 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:20 smithi136 ceph-mon[33038]: fsmap 6 up:standby 2024-04-25T13:17:20.703 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:20 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:17:20.703 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:20 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2542895817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:20.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:20 smithi031 ceph-mon[29457]: Deploying daemon mds.c on smithi136 2024-04-25T13:17:20.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:20 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.136:6834/3086187130,v1:172.21.15.136:6835/3086187130] up:boot 2024-04-25T13:17:20.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:20 smithi031 ceph-mon[29457]: fsmap 6 up:standby 2024-04-25T13:17:20.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:20 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:17:20.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:20 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2542895817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:20.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:20 smithi066 ceph-mon[33137]: Deploying daemon mds.c on smithi136 2024-04-25T13:17:20.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:20 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.136:6834/3086187130,v1:172.21.15.136:6835/3086187130] up:boot 2024-04-25T13:17:20.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:20 smithi066 ceph-mon[33137]: fsmap 6 up:standby 2024-04-25T13:17:20.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:20 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:17:20.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:20 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2542895817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:21.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:21 smithi136 ceph-mon[33038]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:21.719 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 13:17:21 smithi136 systemd[1]: Started Ceph mds.c for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:21.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:21 smithi031 ceph-mon[29457]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:21.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:21 smithi066 ceph-mon[33137]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:22.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.141+0000 7f7b78a35640 1 -- 172.21.15.31:0/32053609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b74107c90 msgr2=0x7f7b7410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.141+0000 7f7b78a35640 1 --2- 172.21.15.31:0/32053609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b74107c90 0x7f7b7410a0e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009ca0 tx=0x7f7b6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.142+0000 7f7b78a35640 1 -- 172.21.15.31:0/32053609 shutdown_connections 2024-04-25T13:17:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.142+0000 7f7b78a35640 1 --2- 172.21.15.31:0/32053609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b74107c90 0x7f7b7410a0e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.142+0000 7f7b78a35640 1 --2- 172.21.15.31:0/32053609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b74105360 0x7f7b74107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.142+0000 7f7b78a35640 1 --2- 172.21.15.31:0/32053609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74102a30 0x7f7b74104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.142+0000 7f7b78a35640 1 -- 172.21.15.31:0/32053609 >> 172.21.15.31:0/32053609 conn(0x7f7b740fc440 msgr2=0x7f7b740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:22.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.142+0000 7f7b78a35640 1 -- 172.21.15.31:0/32053609 shutdown_connections 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.142+0000 7f7b78a35640 1 -- 172.21.15.31:0/32053609 wait complete. 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.142+0000 7f7b78a35640 1 Processor -- start 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.142+0000 7f7b78a35640 1 -- start start 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b78a35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b74102a30 0x7f7b741981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b78a35640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74105360 0x7f7b741986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b78a35640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b74107c90 0x7f7b7419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b78a35640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410daf0 con 0x7f7b74102a30 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b78a35640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410d970 con 0x7f7b74107c90 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b78a35640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410dc70 con 0x7f7b74105360 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b74102a30 0x7f7b741981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b74102a30 0x7f7b741981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48716/0 (socket says 172.21.15.31:48716) 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b7259c640 1 -- 172.21.15.31:0/2099131139 learned_addr learned my addr 172.21.15.31:0/2099131139 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b72d9d640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b74107c90 0x7f7b7419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b71d9b640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74105360 0x7f7b741986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b7259c640 1 -- 172.21.15.31:0/2099131139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74105360 msgr2=0x7f7b741986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b7259c640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74105360 0x7f7b741986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b7259c640 1 -- 172.21.15.31:0/2099131139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b74107c90 msgr2=0x7f7b7419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b7259c640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b74107c90 0x7f7b7419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b7259c640 1 -- 172.21.15.31:0/2099131139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7419fdc0 con 0x7f7b74102a30 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b71d9b640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74105360 0x7f7b741986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b7259c640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b74102a30 0x7f7b741981a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400c890 tx=0x7f7b6400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b637fe640 1 -- 172.21.15.31:0/2099131139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64004220 con 0x7f7b74102a30 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b78a35640 1 -- 172.21.15.31:0/2099131139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b741a00b0 con 0x7f7b74102a30 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.143+0000 7f7b637fe640 1 -- 172.21.15.31:0/2099131139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b640043c0 con 0x7f7b74102a30 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.144+0000 7f7b637fe640 1 -- 172.21.15.31:0/2099131139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64004a00 con 0x7f7b74102a30 2024-04-25T13:17:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.144+0000 7f7b78a35640 1 -- 172.21.15.31:0/2099131139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b741a0600 con 0x7f7b74102a30 2024-04-25T13:17:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.144+0000 7f7b78a35640 1 -- 172.21.15.31:0/2099131139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b740fdf30 con 0x7f7b74102a30 2024-04-25T13:17:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.146+0000 7f7b637fe640 1 -- 172.21.15.31:0/2099131139 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7b640079b0 con 0x7f7b74102a30 2024-04-25T13:17:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.146+0000 7f7b637fe640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b5c0a3bb0 0x7f7b5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.146+0000 7f7b71d9b640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b5c0a3bb0 0x7f7b5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.146+0000 7f7b637fe640 1 -- 172.21.15.31:0/2099131139 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f7b640d00b0 con 0x7f7b74102a30 2024-04-25T13:17:22.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.147+0000 7f7b71d9b640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b5c0a3bb0 0x7f7b5c0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b741996b0 tx=0x7f7b68005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:22.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.150+0000 7f7b637fe640 1 -- 172.21.15.31:0/2099131139 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b64014030 con 0x7f7b74102a30 2024-04-25T13:17:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.377+0000 7f7b78a35640 1 -- 172.21.15.31:0/2099131139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b7402d5c0 con 0x7f7b74102a30 2024-04-25T13:17:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.377+0000 7f7b637fe640 1 -- 172.21.15.31:0/2099131139 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f7b6409cf30 con 0x7f7b74102a30 2024-04-25T13:17:22.378 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:22.378 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T13:17:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 -- 172.21.15.31:0/2099131139 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b5c0a3bb0 msgr2=0x7f7b5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b5c0a3bb0 0x7f7b5c0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b741996b0 tx=0x7f7b68005f90 comp rx=0 tx=0).stop 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 -- 172.21.15.31:0/2099131139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b74102a30 msgr2=0x7f7b741981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b74102a30 0x7f7b741981a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400c890 tx=0x7f7b6400cd50 comp rx=0 tx=0).stop 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 -- 172.21.15.31:0/2099131139 shutdown_connections 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b5c0a3bb0 0x7f7b5c0a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b74107c90 0x7f7b7419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b74105360 0x7f7b741986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 --2- 172.21.15.31:0/2099131139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b74102a30 0x7f7b741981a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 -- 172.21.15.31:0/2099131139 >> 172.21.15.31:0/2099131139 conn(0x7f7b740fc440 msgr2=0x7f7b74103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 -- 172.21.15.31:0/2099131139 shutdown_connections 2024-04-25T13:17:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:22.379+0000 7f7b78a35640 1 -- 172.21.15.31:0/2099131139 wait complete. 2024-04-25T13:17:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: Deploying daemon mds.h on smithi066 2024-04-25T13:17:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2099131139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.136:6836/3062184830,v1:172.21.15.136:6837/3062184830] up:boot 2024-04-25T13:17:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: fsmap 7 up:standby 2024-04-25T13:17:22.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:22 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:17:22.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:22.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:22.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:22.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:22.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:22.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:22.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: Deploying daemon mds.h on smithi066 2024-04-25T13:17:22.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:22.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2099131139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:22.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.136:6836/3062184830,v1:172.21.15.136:6837/3062184830] up:boot 2024-04-25T13:17:22.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: fsmap 7 up:standby 2024-04-25T13:17:22.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:22 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:17:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:22.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:22.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: Deploying daemon mds.h on smithi066 2024-04-25T13:17:22.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:22.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2099131139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:22.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.136:6836/3062184830,v1:172.21.15.136:6837/3062184830] up:boot 2024-04-25T13:17:22.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: fsmap 7 up:standby 2024-04-25T13:17:22.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:22 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:17:24.094 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:17:23 smithi066 systemd[1]: Starting Ceph mds.h for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:17:24.376 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:17:24 smithi066 podman[63316]: 2024-04-25 13:17:23.996389616 +0000 UTC m=+0.018386277 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:24.376 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:17:24 smithi066 podman[63316]: 2024-04-25 13:17:24.115056338 +0000 UTC m=+0.137053001 container create b9d5d6c3c08be5d4e0148deab4eeb4a102abc7186bfe1bbc8ec0ee62bbda3f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T13:17:24.376 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:17:24 smithi066 podman[63316]: 2024-04-25 13:17:24.376213249 +0000 UTC m=+0.398209912 container init b9d5d6c3c08be5d4e0148deab4eeb4a102abc7186bfe1bbc8ec0ee62bbda3f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T13:17:24.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.506+0000 7f9069f8a640 1 -- 172.21.15.31:0/1455645260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90641029a0 msgr2=0x7f90641092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.506+0000 7f9069f8a640 1 --2- 172.21.15.31:0/1455645260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90641029a0 0x7f90641092c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f905c009ca0 tx=0x7f905c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.506+0000 7f9069f8a640 1 -- 172.21.15.31:0/1455645260 shutdown_connections 2024-04-25T13:17:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.506+0000 7f9069f8a640 1 --2- 172.21.15.31:0/1455645260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90641029a0 0x7f90641092c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.506+0000 7f9069f8a640 1 --2- 172.21.15.31:0/1455645260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064101fe0 0x7f9064102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.506+0000 7f9069f8a640 1 --2- 172.21.15.31:0/1455645260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9064100e90 0x7f90641012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.506+0000 7f9069f8a640 1 -- 172.21.15.31:0/1455645260 >> 172.21.15.31:0/1455645260 conn(0x7f90640fc400 msgr2=0x7f90640fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.506+0000 7f9069f8a640 1 -- 172.21.15.31:0/1455645260 shutdown_connections 2024-04-25T13:17:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.506+0000 7f9069f8a640 1 -- 172.21.15.31:0/1455645260 wait complete. 2024-04-25T13:17:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.507+0000 7f9069f8a640 1 Processor -- start 2024-04-25T13:17:24.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.507+0000 7f9069f8a640 1 -- start start 2024-04-25T13:17:24.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f9069f8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064100e90 0x7f9064198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:24.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f9069f8a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064101fe0 0x7f9064198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:24.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f9069f8a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90641029a0 0x7f906419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:24.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f9069f8a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f906410ca30 con 0x7f9064100e90 2024-04-25T13:17:24.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f9069f8a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f906410c8b0 con 0x7f90641029a0 2024-04-25T13:17:24.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f9069f8a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f906410cbb0 con 0x7f9064101fe0 2024-04-25T13:17:24.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f90637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064100e90 0x7f9064198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:24.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f90637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064100e90 0x7f9064198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48728/0 (socket says 172.21.15.31:48728) 2024-04-25T13:17:24.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f90637fe640 1 -- 172.21.15.31:0/847688382 learned_addr learned my addr 172.21.15.31:0/847688382 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:24.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f9062ffd640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064101fe0 0x7f9064198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:24.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f9063fff640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90641029a0 0x7f906419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:24.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f90637fe640 1 -- 172.21.15.31:0/847688382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064101fe0 msgr2=0x7f9064198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:24.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f90637fe640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064101fe0 0x7f9064198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f90637fe640 1 -- 172.21.15.31:0/847688382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90641029a0 msgr2=0x7f906419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:24.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f90637fe640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90641029a0 0x7f906419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f90637fe640 1 -- 172.21.15.31:0/847688382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906419fd70 con 0x7f9064100e90 2024-04-25T13:17:24.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.508+0000 7f90637fe640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064100e90 0x7f9064198150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f905400e9f0 tx=0x7f905400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:24.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.511+0000 7f9060ff9640 1 -- 172.21.15.31:0/847688382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905400cd80 con 0x7f9064100e90 2024-04-25T13:17:24.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.511+0000 7f9069f8a640 1 -- 172.21.15.31:0/847688382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90641a0060 con 0x7f9064100e90 2024-04-25T13:17:24.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.511+0000 7f9069f8a640 1 -- 172.21.15.31:0/847688382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90641a0600 con 0x7f9064100e90 2024-04-25T13:17:24.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.511+0000 7f9060ff9640 1 -- 172.21.15.31:0/847688382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90540108e0 con 0x7f9064100e90 2024-04-25T13:17:24.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.511+0000 7f9060ff9640 1 -- 172.21.15.31:0/847688382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90540188e0 con 0x7f9064100e90 2024-04-25T13:17:24.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.512+0000 7f9069f8a640 1 -- 172.21.15.31:0/847688382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90500052c0 con 0x7f9064100e90 2024-04-25T13:17:24.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.513+0000 7f9060ff9640 1 -- 172.21.15.31:0/847688382 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9054010430 con 0x7f9064100e90 2024-04-25T13:17:24.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.514+0000 7f9060ff9640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f904c0a3bb0 0x7f904c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:24.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.514+0000 7f9060ff9640 1 -- 172.21.15.31:0/847688382 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f90540d01f0 con 0x7f9064100e90 2024-04-25T13:17:24.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.514+0000 7f9062ffd640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f904c0a3bb0 0x7f904c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:24.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.514+0000 7f9062ffd640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f904c0a3bb0 0x7f904c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9064199660 tx=0x7f9058005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:24.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.521+0000 7f9060ff9640 1 -- 172.21.15.31:0/847688382 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f905409ca20 con 0x7f9064100e90 2024-04-25T13:17:24.658 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:17:24 smithi066 podman[63316]: 2024-04-25 13:17:24.379454667 +0000 UTC m=+0.401451332 container start b9d5d6c3c08be5d4e0148deab4eeb4a102abc7186bfe1bbc8ec0ee62bbda3f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T13:17:24.658 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:17:24 smithi066 ceph-mds[63335]: -- 172.21.15.66:0/1593901070 <== mon.2 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56208e0a5a40 con 0x56208e078c00 2024-04-25T13:17:24.658 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:17:24 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-h[63331]: starting mds.h at 2024-04-25T13:17:24.658 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:17:24 smithi066 bash[63316]: b9d5d6c3c08be5d4e0148deab4eeb4a102abc7186bfe1bbc8ec0ee62bbda3f0e 2024-04-25T13:17:24.658 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:17:24 smithi066 systemd[1]: Started Ceph mds.h for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:24.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.748+0000 7f9069f8a640 1 -- 172.21.15.31:0/847688382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9050005880 con 0x7f9064100e90 2024-04-25T13:17:24.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.749+0000 7f9060ff9640 1 -- 172.21.15.31:0/847688382 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f90540a0d20 con 0x7f9064100e90 2024-04-25T13:17:24.749 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:24.749 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:17:24.749 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2024-04-25T13:17:24.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 -- 172.21.15.31:0/847688382 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f904c0a3bb0 msgr2=0x7f904c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:24.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f904c0a3bb0 0x7f904c0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9064199660 tx=0x7f9058005f90 comp rx=0 tx=0).stop 2024-04-25T13:17:24.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 -- 172.21.15.31:0/847688382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064100e90 msgr2=0x7f9064198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:24.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064100e90 0x7f9064198150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f905400e9f0 tx=0x7f905400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 -- 172.21.15.31:0/847688382 shutdown_connections 2024-04-25T13:17:24.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f904c0a3bb0 0x7f904c0a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f90641029a0 0x7f906419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064101fe0 0x7f9064198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 --2- 172.21.15.31:0/847688382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064100e90 0x7f9064198150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 -- 172.21.15.31:0/847688382 >> 172.21.15.31:0/847688382 conn(0x7f90640fc400 msgr2=0x7f90640690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:24.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 -- 172.21.15.31:0/847688382 shutdown_connections 2024-04-25T13:17:24.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:24.751+0000 7f9069f8a640 1 -- 172.21.15.31:0/847688382 wait complete. 2024-04-25T13:17:25.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:25 smithi066 ceph-mon[33137]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:25.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:25 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:25.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:25 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:25.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:25 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:25.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:25 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:25.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:25 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:25.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:25 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:25.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:25 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/847688382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:25.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:25 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.66:6836/23034509,v1:172.21.15.66:6837/23034509] up:boot 2024-04-25T13:17:25.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:25 smithi066 ceph-mon[33137]: fsmap 8 up:standby 2024-04-25T13:17:25.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:25 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:17:25.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:25 smithi136 ceph-mon[33038]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:25.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:25 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:25.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:25 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:25.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:25 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:25.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:25 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:25.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:25 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:25.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:25 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:25.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:25 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/847688382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:25.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:25 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.66:6836/23034509,v1:172.21.15.66:6837/23034509] up:boot 2024-04-25T13:17:25.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:25 smithi136 ceph-mon[33038]: fsmap 8 up:standby 2024-04-25T13:17:25.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:25 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:17:25.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:25 smithi031 ceph-mon[29457]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:25.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:25 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:25.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:25 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:25.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:25 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:25.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:25 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:25.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:25 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:25.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:25 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:25.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:25 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/847688382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:25.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:25 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.66:6836/23034509,v1:172.21.15.66:6837/23034509] up:boot 2024-04-25T13:17:25.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:25 smithi031 ceph-mon[29457]: fsmap 8 up:standby 2024-04-25T13:17:25.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:25 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:17:26.763 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:26.787 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:26 smithi031 ceph-mon[29457]: Deploying daemon mds.i on smithi136 2024-04-25T13:17:26.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.870+0000 7f40e0ae1640 1 -- 172.21.15.31:0/2354431159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40dc102040 msgr2=0x7f40dc1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:26.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.870+0000 7f40e0ae1640 1 --2- 172.21.15.31:0/2354431159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40dc102040 0x7f40dc1024c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009900 tx=0x7f40d002a410 comp rx=0 tx=0).stop 2024-04-25T13:17:26.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.870+0000 7f40e0ae1640 1 -- 172.21.15.31:0/2354431159 shutdown_connections 2024-04-25T13:17:26.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.870+0000 7f40e0ae1640 1 --2- 172.21.15.31:0/2354431159 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc102a00 0x7f40dc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.870+0000 7f40e0ae1640 1 --2- 172.21.15.31:0/2354431159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40dc102040 0x7f40dc1024c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.870+0000 7f40e0ae1640 1 --2- 172.21.15.31:0/2354431159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40dc100e60 0x7f40dc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.870+0000 7f40e0ae1640 1 -- 172.21.15.31:0/2354431159 >> 172.21.15.31:0/2354431159 conn(0x7f40dc0fc440 msgr2=0x7f40dc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.870+0000 7f40e0ae1640 1 -- 172.21.15.31:0/2354431159 shutdown_connections 2024-04-25T13:17:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.870+0000 7f40e0ae1640 1 -- 172.21.15.31:0/2354431159 wait complete. 2024-04-25T13:17:26.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40e0ae1640 1 Processor -- start 2024-04-25T13:17:26.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40e0ae1640 1 -- start start 2024-04-25T13:17:26.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40e0ae1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40dc100e60 0x7f40dc06b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:26.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40e0ae1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc102040 0x7f40dc19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:26.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40e0ae1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40dc102a00 0x7f40dc19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:26.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40e0ae1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10ca10 con 0x7f40dc102a00 2024-04-25T13:17:26.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40e0ae1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10c890 con 0x7f40dc102040 2024-04-25T13:17:26.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40e0ae1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10cb90 con 0x7f40dc100e60 2024-04-25T13:17:26.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40dad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40dc102a00 0x7f40dc19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:26.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40da59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40dc100e60 0x7f40dc06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:26.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40da59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40dc100e60 0x7f40dc06b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.31:59968/0 (socket says 172.21.15.31:59968) 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40da59c640 1 -- 172.21.15.31:0/1735460148 learned_addr learned my addr 172.21.15.31:0/1735460148 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40dad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40dc102a00 0x7f40dc19f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48752/0 (socket says 172.21.15.31:48752) 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40d9d9b640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc102040 0x7f40dc19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40da59c640 1 -- 172.21.15.31:0/1735460148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc102040 msgr2=0x7f40dc19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40da59c640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc102040 0x7f40dc19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40da59c640 1 -- 172.21.15.31:0/1735460148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40dc102a00 msgr2=0x7f40dc19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40da59c640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40dc102a00 0x7f40dc19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40da59c640 1 -- 172.21.15.31:0/1735460148 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc19fe90 con 0x7f40dc100e60 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40dad9d640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40dc102a00 0x7f40dc19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.871+0000 7f40da59c640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40dc100e60 0x7f40dc06b120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00e9c0 tx=0x7f40cc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.872+0000 7f40cb7fe640 1 -- 172.21.15.31:0/1735460148 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc00cd80 con 0x7f40dc100e60 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.872+0000 7f40e0ae1640 1 -- 172.21.15.31:0/1735460148 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40dc1a0120 con 0x7f40dc100e60 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.872+0000 7f40cb7fe640 1 -- 172.21.15.31:0/1735460148 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40cc0035f0 con 0x7f40dc100e60 2024-04-25T13:17:26.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.872+0000 7f40cb7fe640 1 -- 172.21.15.31:0/1735460148 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc010690 con 0x7f40dc100e60 2024-04-25T13:17:26.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.872+0000 7f40e0ae1640 1 -- 172.21.15.31:0/1735460148 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40dc1a0670 con 0x7f40dc100e60 2024-04-25T13:17:26.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.873+0000 7f40e0ae1640 1 -- 172.21.15.31:0/1735460148 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40dc0691c0 con 0x7f40dc100e60 2024-04-25T13:17:26.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.874+0000 7f40cb7fe640 1 -- 172.21.15.31:0/1735460148 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f40cc004000 con 0x7f40dc100e60 2024-04-25T13:17:26.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.874+0000 7f40cb7fe640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40c40a3bb0 0x7f40c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:26.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.875+0000 7f40d9d9b640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40c40a3bb0 0x7f40c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:26.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.875+0000 7f40cb7fe640 1 -- 172.21.15.31:0/1735460148 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f40cc0d06c0 con 0x7f40dc100e60 2024-04-25T13:17:26.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.875+0000 7f40d9d9b640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40c40a3bb0 0x7f40c40a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f40dc06ae90 tx=0x7f40d00314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:26.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:26.878+0000 7f40cb7fe640 1 -- 172.21.15.31:0/1735460148 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40cc09d4a0 con 0x7f40dc100e60 2024-04-25T13:17:26.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:26 smithi066 ceph-mon[33137]: Deploying daemon mds.i on smithi136 2024-04-25T13:17:26.917 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:17:26 smithi136 systemd[1]: Starting Ceph mds.i for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:17:26.918 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:26 smithi136 ceph-mon[33038]: Deploying daemon mds.i on smithi136 2024-04-25T13:17:27.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.104+0000 7f40e0ae1640 1 -- 172.21.15.31:0/1735460148 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40dc1082d0 con 0x7f40dc100e60 2024-04-25T13:17:27.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.118+0000 7f40cb7fe640 1 -- 172.21.15.31:0/1735460148 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7f40cc09cda0 con 0x7f40dc100e60 2024-04-25T13:17:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:27.118 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:17:27.118 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2024-04-25T13:17:27.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 -- 172.21.15.31:0/1735460148 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40c40a3bb0 msgr2=0x7f40c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:27.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40c40a3bb0 0x7f40c40a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f40dc06ae90 tx=0x7f40d00314a0 comp rx=0 tx=0).stop 2024-04-25T13:17:27.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 -- 172.21.15.31:0/1735460148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40dc100e60 msgr2=0x7f40dc06b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:27.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40dc100e60 0x7f40dc06b120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00e9c0 tx=0x7f40cc00ee80 comp rx=0 tx=0).stop 2024-04-25T13:17:27.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 -- 172.21.15.31:0/1735460148 shutdown_connections 2024-04-25T13:17:27.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40c40a3bb0 0x7f40c40a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40dc102a00 0x7f40dc19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40dc102040 0x7f40dc19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 --2- 172.21.15.31:0/1735460148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40dc100e60 0x7f40dc06b120 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 -- 172.21.15.31:0/1735460148 >> 172.21.15.31:0/1735460148 conn(0x7f40dc0fc440 msgr2=0x7f40dc1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 -- 172.21.15.31:0/1735460148 shutdown_connections 2024-04-25T13:17:27.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:27.120+0000 7f40e0ae1640 1 -- 172.21.15.31:0/1735460148 wait complete. 2024-04-25T13:17:27.169 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:17:26 smithi136 podman[63628]: 2024-04-25 13:17:26.820806435 +0000 UTC m=+0.018086393 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:27.169 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:17:26 smithi136 podman[63628]: 2024-04-25 13:17:26.950266689 +0000 UTC m=+0.147546665 container create 7ba737bdf5b1a6bdb9a14727625dfd8ec380af7ad373f193d399951c90e5ae4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T13:17:27.169 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:17:27 smithi136 podman[63628]: 2024-04-25 13:17:27.169468766 +0000 UTC m=+0.366748700 container init 7ba737bdf5b1a6bdb9a14727625dfd8ec380af7ad373f193d399951c90e5ae4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T13:17:27.469 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:17:27 smithi136 podman[63628]: 2024-04-25 13:17:27.172296716 +0000 UTC m=+0.369576651 container start 7ba737bdf5b1a6bdb9a14727625dfd8ec380af7ad373f193d399951c90e5ae4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T13:17:27.469 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:17:27 smithi136 ceph-mds[63647]: -- 172.21.15.136:0/1018706924 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558433ab3a40 con 0x558433a86c00 2024-04-25T13:17:27.469 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:17:27 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i[63643]: starting mds.i at 2024-04-25T13:17:27.469 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:17:27 smithi136 bash[63628]: 7ba737bdf5b1a6bdb9a14727625dfd8ec380af7ad373f193d399951c90e5ae4d 2024-04-25T13:17:27.469 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:17:27 smithi136 systemd[1]: Started Ceph mds.i for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:27.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:27 smithi066 ceph-mon[33137]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:27.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:27 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1735460148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:27.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:27.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:27.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:27.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:27.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:27.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:27 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:27.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:27 smithi136 ceph-mon[33038]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:27.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:27 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1735460148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:27.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:27.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:27.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:27.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:27.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:27.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:27 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:27.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:27 smithi031 ceph-mon[29457]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:27.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:27 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1735460148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:27.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:27.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:27.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:27.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:27.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:27.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:27 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:28.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:28 smithi031 ceph-mon[29457]: Deploying daemon mds.g on smithi031 2024-04-25T13:17:28.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:28 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] up:boot 2024-04-25T13:17:28.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:28 smithi031 ceph-mon[29457]: fsmap 9 up:standby 2024-04-25T13:17:28.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:28 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:17:28.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:28 smithi066 ceph-mon[33137]: Deploying daemon mds.g on smithi031 2024-04-25T13:17:28.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:28 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] up:boot 2024-04-25T13:17:28.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:28 smithi066 ceph-mon[33137]: fsmap 9 up:standby 2024-04-25T13:17:28.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:28 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:17:28.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:28 smithi136 ceph-mon[33038]: Deploying daemon mds.g on smithi031 2024-04-25T13:17:28.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:28 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] up:boot 2024-04-25T13:17:28.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:28 smithi136 ceph-mon[33038]: fsmap 9 up:standby 2024-04-25T13:17:28.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:28 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:17:29.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.290+0000 7f9613413640 1 -- 172.21.15.31:0/1058390291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960c107d30 msgr2=0x7f960c10a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.290+0000 7f9613413640 1 --2- 172.21.15.31:0/1058390291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960c107d30 0x7f960c10a120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9604009ca0 tx=0x7f960402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.290+0000 7f9613413640 1 -- 172.21.15.31:0/1058390291 shutdown_connections 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.290+0000 7f9613413640 1 --2- 172.21.15.31:0/1058390291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960c107d30 0x7f960c10a120 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.290+0000 7f9613413640 1 --2- 172.21.15.31:0/1058390291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f960c105400 0x7f960c1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.290+0000 7f9613413640 1 --2- 172.21.15.31:0/1058390291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960c102ad0 0x7f960c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.290+0000 7f9613413640 1 -- 172.21.15.31:0/1058390291 >> 172.21.15.31:0/1058390291 conn(0x7f960c0fc440 msgr2=0x7f960c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.290+0000 7f9613413640 1 -- 172.21.15.31:0/1058390291 shutdown_connections 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.290+0000 7f9613413640 1 -- 172.21.15.31:0/1058390291 wait complete. 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f9613413640 1 Processor -- start 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f9613413640 1 -- start start 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f9613413640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960c102ad0 0x7f960c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f9613413640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960c105400 0x7f960c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f9613413640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f960c107d30 0x7f960c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f9613413640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10db70 con 0x7f960c105400 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f9613413640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10d9f0 con 0x7f960c102ad0 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f9613413640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10dcf0 con 0x7f960c107d30 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f96109ae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960c105400 0x7f960c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f96109ae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960c105400 0x7f960c1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48782/0 (socket says 172.21.15.31:48782) 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f96109ae640 1 -- 172.21.15.31:0/2755429313 learned_addr learned my addr 172.21.15.31:0/2755429313 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f96119b0640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f960c107d30 0x7f960c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f96111af640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960c102ad0 0x7f960c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f96109ae640 1 -- 172.21.15.31:0/2755429313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f960c107d30 msgr2=0x7f960c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f96109ae640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f960c107d30 0x7f960c19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f96109ae640 1 -- 172.21.15.31:0/2755429313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960c102ad0 msgr2=0x7f960c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f96109ae640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960c102ad0 0x7f960c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f96109ae640 1 -- 172.21.15.31:0/2755429313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c19fe90 con 0x7f960c105400 2024-04-25T13:17:29.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.291+0000 7f96111af640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960c102ad0 0x7f960c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.292+0000 7f96109ae640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960c105400 0x7f960c1987b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f960000b450 tx=0x7f960000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.292+0000 7f95fe7fc640 1 -- 172.21.15.31:0/2755429313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600015020 con 0x7f960c105400 2024-04-25T13:17:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.292+0000 7f95fe7fc640 1 -- 172.21.15.31:0/2755429313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9600011d40 con 0x7f960c105400 2024-04-25T13:17:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.292+0000 7f95fe7fc640 1 -- 172.21.15.31:0/2755429313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960000f340 con 0x7f960c105400 2024-04-25T13:17:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.292+0000 7f9613413640 1 -- 172.21.15.31:0/2755429313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c1a0180 con 0x7f960c105400 2024-04-25T13:17:29.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.292+0000 7f9613413640 1 -- 172.21.15.31:0/2755429313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c1a0700 con 0x7f960c105400 2024-04-25T13:17:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.293+0000 7f9613413640 1 -- 172.21.15.31:0/2755429313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960c0fdf30 con 0x7f960c105400 2024-04-25T13:17:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.294+0000 7f95fe7fc640 1 -- 172.21.15.31:0/2755429313 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9600018020 con 0x7f960c105400 2024-04-25T13:17:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.295+0000 7f95fe7fc640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f95f80a3bb0 0x7f95f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.295+0000 7f96111af640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f95f80a3bb0 0x7f95f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.295+0000 7f95fe7fc640 1 -- 172.21.15.31:0/2755429313 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f96000d0160 con 0x7f960c105400 2024-04-25T13:17:29.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.296+0000 7f96111af640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f95f80a3bb0 0x7f95f80a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9608009b40 tx=0x7f9608005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.299+0000 7f95fe7fc640 1 -- 172.21.15.31:0/2755429313 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f960001f020 con 0x7f960c105400 2024-04-25T13:17:29.483 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:17:29 smithi031 systemd[1]: Starting Ceph mds.g for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:17:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.531+0000 7f9613413640 1 -- 172.21.15.31:0/2755429313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f960c02d810 con 0x7f960c105400 2024-04-25T13:17:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.532+0000 7f95fe7fc640 1 -- 172.21.15.31:0/2755429313 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7f960000d040 con 0x7f960c105400 2024-04-25T13:17:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:29.532 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:17:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 10 2024-04-25T13:17:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 -- 172.21.15.31:0/2755429313 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f95f80a3bb0 msgr2=0x7f95f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f95f80a3bb0 0x7f95f80a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9608009b40 tx=0x7f9608005e50 comp rx=0 tx=0).stop 2024-04-25T13:17:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 -- 172.21.15.31:0/2755429313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960c105400 msgr2=0x7f960c1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960c105400 0x7f960c1987b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f960000b450 tx=0x7f960000b910 comp rx=0 tx=0).stop 2024-04-25T13:17:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 -- 172.21.15.31:0/2755429313 shutdown_connections 2024-04-25T13:17:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f95f80a3bb0 0x7f95f80a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f960c107d30 0x7f960c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f960c105400 0x7f960c1987b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 --2- 172.21.15.31:0/2755429313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f960c102ad0 0x7f960c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 -- 172.21.15.31:0/2755429313 >> 172.21.15.31:0/2755429313 conn(0x7f960c0fc440 msgr2=0x7f960c103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 -- 172.21.15.31:0/2755429313 shutdown_connections 2024-04-25T13:17:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:29.534+0000 7f9613413640 1 -- 172.21.15.31:0/2755429313 wait complete. 2024-04-25T13:17:29.861 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:29 smithi031 ceph-mon[29457]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:29.861 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:29 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2755429313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:29.861 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:17:29 smithi031 podman[70883]: 2024-04-25 13:17:29.561387881 +0000 UTC m=+0.022335502 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:29.861 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:17:29 smithi031 podman[70883]: 2024-04-25 13:17:29.663905387 +0000 UTC m=+0.124852988 container create f21d230cf9317af5bc8cfe8d0577c41c0c36c2f98aa3e7c375cbceebd3b773b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-g, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:17:29.869 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:29 smithi136 ceph-mon[33038]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:29.869 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:29 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2755429313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:29.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:29 smithi066 ceph-mon[33137]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:29.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:29 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2755429313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:30.234 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:17:29 smithi031 podman[70883]: 2024-04-25 13:17:29.958357937 +0000 UTC m=+0.419305539 container init f21d230cf9317af5bc8cfe8d0577c41c0c36c2f98aa3e7c375cbceebd3b773b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T13:17:30.234 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:17:29 smithi031 podman[70883]: 2024-04-25 13:17:29.96179158 +0000 UTC m=+0.422739181 container start f21d230cf9317af5bc8cfe8d0577c41c0c36c2f98aa3e7c375cbceebd3b773b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:17:30.235 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:17:29 smithi031 ceph-mds[70903]: -- 172.21.15.31:0/4244976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557983339a40 con 0x55798330d000 2024-04-25T13:17:30.235 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:17:29 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-g[70899]: starting mds.g at 2024-04-25T13:17:30.235 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:17:30 smithi031 bash[70883]: f21d230cf9317af5bc8cfe8d0577c41c0c36c2f98aa3e7c375cbceebd3b773b6 2024-04-25T13:17:30.235 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:17:30 smithi031 systemd[1]: Started Ceph mds.g for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:31 smithi066 ceph-mon[33137]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:31 smithi066 ceph-mon[33137]: Deploying daemon mds.a on smithi031 2024-04-25T13:17:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:31 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.31:6838/2599371925,v1:172.21.15.31:6839/2599371925] up:boot 2024-04-25T13:17:31.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:31 smithi066 ceph-mon[33137]: fsmap 10 up:standby 2024-04-25T13:17:31.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:17:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:31 smithi136 ceph-mon[33038]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:31 smithi136 ceph-mon[33038]: Deploying daemon mds.a on smithi031 2024-04-25T13:17:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:31 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.31:6838/2599371925,v1:172.21.15.31:6839/2599371925] up:boot 2024-04-25T13:17:31.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:31 smithi136 ceph-mon[33038]: fsmap 10 up:standby 2024-04-25T13:17:31.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:17:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:31 smithi031 ceph-mon[29457]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:31 smithi031 ceph-mon[29457]: Deploying daemon mds.a on smithi031 2024-04-25T13:17:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:31 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.31:6838/2599371925,v1:172.21.15.31:6839/2599371925] up:boot 2024-04-25T13:17:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:31 smithi031 ceph-mon[29457]: fsmap 10 up:standby 2024-04-25T13:17:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:17:31.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:31.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.665+0000 7f1b1af19640 1 -- 172.21.15.31:0/2665285999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b14101fc0 msgr2=0x7f1b141023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:31.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.665+0000 7f1b1af19640 1 --2- 172.21.15.31:0/2665285999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b14101fc0 0x7f1b141023e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009900 tx=0x7f1b0402a410 comp rx=0 tx=0).stop 2024-04-25T13:17:31.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.665+0000 7f1b1af19640 1 -- 172.21.15.31:0/2665285999 shutdown_connections 2024-04-25T13:17:31.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.665+0000 7f1b1af19640 1 --2- 172.21.15.31:0/2665285999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b14103b70 0x7f1b1410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.665+0000 7f1b1af19640 1 --2- 172.21.15.31:0/2665285999 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b141031b0 0x7f1b14103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.665+0000 7f1b1af19640 1 --2- 172.21.15.31:0/2665285999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b14101fc0 0x7f1b141023e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.665+0000 7f1b1af19640 1 -- 172.21.15.31:0/2665285999 >> 172.21.15.31:0/2665285999 conn(0x7f1b140fd550 msgr2=0x7f1b140ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:31.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.665+0000 7f1b1af19640 1 -- 172.21.15.31:0/2665285999 shutdown_connections 2024-04-25T13:17:31.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.665+0000 7f1b1af19640 1 -- 172.21.15.31:0/2665285999 wait complete. 2024-04-25T13:17:31.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b1af19640 1 Processor -- start 2024-04-25T13:17:31.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b1af19640 1 -- start start 2024-04-25T13:17:31.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b1af19640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b14101fc0 0x7f1b14075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:31.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b1af19640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b141031b0 0x7f1b14074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:31.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b1af19640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b14103b70 0x7f1b14074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:31.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b1af19640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1410db10 con 0x7f1b14101fc0 2024-04-25T13:17:31.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b1af19640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1410d990 con 0x7f1b141031b0 2024-04-25T13:17:31.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b1af19640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1410dc90 con 0x7f1b14103b70 2024-04-25T13:17:31.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b18cb5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b14101fc0 0x7f1b14075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:31.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b18cb5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b14101fc0 0x7f1b14075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36778/0 (socket says 172.21.15.31:36778) 2024-04-25T13:17:31.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b18cb5640 1 -- 172.21.15.31:0/940235989 learned_addr learned my addr 172.21.15.31:0/940235989 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:31.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b13fff640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b141031b0 0x7f1b14074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:31.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.666+0000 7f1b194b6640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b14103b70 0x7f1b14074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:31.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b18cb5640 1 -- 172.21.15.31:0/940235989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b14103b70 msgr2=0x7f1b14074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:31.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b18cb5640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b14103b70 0x7f1b14074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b18cb5640 1 -- 172.21.15.31:0/940235989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b141031b0 msgr2=0x7f1b14074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:31.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b18cb5640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b141031b0 0x7f1b14074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b18cb5640 1 -- 172.21.15.31:0/940235989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b14075110 con 0x7f1b14101fc0 2024-04-25T13:17:31.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b194b6640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b14103b70 0x7f1b14074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:17:31.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b18cb5640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b14101fc0 0x7f1b14075d00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1b04031500 tx=0x7f1b04003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:31.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b11ffb640 1 -- 172.21.15.31:0/940235989 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0403f070 con 0x7f1b14101fc0 2024-04-25T13:17:31.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b11ffb640 1 -- 172.21.15.31:0/940235989 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b04003e70 con 0x7f1b14101fc0 2024-04-25T13:17:31.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b1af19640 1 -- 172.21.15.31:0/940235989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b141a22f0 con 0x7f1b14101fc0 2024-04-25T13:17:31.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b11ffb640 1 -- 172.21.15.31:0/940235989 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04035030 con 0x7f1b14101fc0 2024-04-25T13:17:31.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.667+0000 7f1b1af19640 1 -- 172.21.15.31:0/940235989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b141a2840 con 0x7f1b14101fc0 2024-04-25T13:17:31.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.668+0000 7f1b1af19640 1 -- 172.21.15.31:0/940235989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b14109400 con 0x7f1b14101fc0 2024-04-25T13:17:31.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.669+0000 7f1b11ffb640 1 -- 172.21.15.31:0/940235989 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1b04042070 con 0x7f1b14101fc0 2024-04-25T13:17:31.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.670+0000 7f1b11ffb640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1b000a3bb0 0x7f1b000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:31.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.670+0000 7f1b11ffb640 1 -- 172.21.15.31:0/940235989 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f1b040f0be0 con 0x7f1b14101fc0 2024-04-25T13:17:31.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.670+0000 7f1b13fff640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1b000a3bb0 0x7f1b000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:31.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.671+0000 7f1b13fff640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1b000a3bb0 0x7f1b000a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b14075a70 tx=0x7f1afc00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:31.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.675+0000 7f1b11ffb640 1 -- 172.21.15.31:0/940235989 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b040bda60 con 0x7f1b14101fc0 2024-04-25T13:17:31.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.911+0000 7f1b1af19640 1 -- 172.21.15.31:0/940235989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b1402d5c0 con 0x7f1b14101fc0 2024-04-25T13:17:31.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.912+0000 7f1b11ffb640 1 -- 172.21.15.31:0/940235989 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7f1b040bd360 con 0x7f1b14101fc0 2024-04-25T13:17:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2599371925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2599371925},{"type":"v1","addr":"172.21.15.31:6839","nonce":2599371925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:17:31.914 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 11 2024-04-25T13:17:31.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 -- 172.21.15.31:0/940235989 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1b000a3bb0 msgr2=0x7f1b000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:31.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1b000a3bb0 0x7f1b000a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b14075a70 tx=0x7f1afc00a2d0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 -- 172.21.15.31:0/940235989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b14101fc0 msgr2=0x7f1b14075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:31.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b14101fc0 0x7f1b14075d00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1b04031500 tx=0x7f1b04003c80 comp rx=0 tx=0).stop 2024-04-25T13:17:31.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 -- 172.21.15.31:0/940235989 shutdown_connections 2024-04-25T13:17:31.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1b000a3bb0 0x7f1b000a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b14103b70 0x7f1b14074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b141031b0 0x7f1b14074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 --2- 172.21.15.31:0/940235989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b14101fc0 0x7f1b14075d00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 -- 172.21.15.31:0/940235989 >> 172.21.15.31:0/940235989 conn(0x7f1b140fd550 msgr2=0x7f1b141065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:31.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 -- 172.21.15.31:0/940235989 shutdown_connections 2024-04-25T13:17:31.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:31.915+0000 7f1b1af19640 1 -- 172.21.15.31:0/940235989 wait complete. 2024-04-25T13:17:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:32 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/940235989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:32.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:32 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/940235989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:32.469 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:32 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/940235989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:32.470 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:17:32 smithi031 systemd[1]: Starting Ceph mds.a for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:17:32.734 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:17:32 smithi031 podman[71233]: 2024-04-25 13:17:32.372956358 +0000 UTC m=+0.018642153 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:17:32.734 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:17:32 smithi031 podman[71233]: 2024-04-25 13:17:32.495491382 +0000 UTC m=+0.141177150 container create 8dcf8d9c8c9180f82af7af6c9e047aa3be7126e19ea08b792d7625d51f1a4ce1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T13:17:32.734 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:17:32 smithi031 podman[71233]: 2024-04-25 13:17:32.715464486 +0000 UTC m=+0.361150263 container init 8dcf8d9c8c9180f82af7af6c9e047aa3be7126e19ea08b792d7625d51f1a4ce1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T13:17:32.734 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:17:32 smithi031 podman[71233]: 2024-04-25 13:17:32.71818337 +0000 UTC m=+0.363869140 container start 8dcf8d9c8c9180f82af7af6c9e047aa3be7126e19ea08b792d7625d51f1a4ce1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-04-25T13:17:33.121 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:17:32 smithi031 ceph-mds[71252]: -- 172.21.15.31:0/2224775697 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604b3055a40 con 0x5604b3028800 2024-04-25T13:17:33.121 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:17:32 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-a[71248]: starting mds.a at 2024-04-25T13:17:33.121 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:17:32 smithi031 bash[71233]: 8dcf8d9c8c9180f82af7af6c9e047aa3be7126e19ea08b792d7625d51f1a4ce1 2024-04-25T13:17:33.121 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:17:32 smithi031 systemd[1]: Started Ceph mds.a for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:33 smithi066 ceph-mon[33137]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:33 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:33 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:33 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:33 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:33 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:33 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:33 smithi136 ceph-mon[33038]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:33 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:33.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:33 smithi031 ceph-mon[29457]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:33.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:17:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:17:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:33 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:33.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.046+0000 7f40ef5ad640 1 -- 172.21.15.31:0/3954531814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e8100820 msgr2=0x7f40e8102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.046+0000 7f40ef5ad640 1 --2- 172.21.15.31:0/3954531814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e8100820 0x7f40e8102c50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009ca0 tx=0x7f40e402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.046+0000 7f40ef5ad640 1 -- 172.21.15.31:0/3954531814 shutdown_connections 2024-04-25T13:17:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.046+0000 7f40ef5ad640 1 --2- 172.21.15.31:0/3954531814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e8105b40 0x7f40e8105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.046+0000 7f40ef5ad640 1 --2- 172.21.15.31:0/3954531814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40e8103190 0x7f40e8105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.046+0000 7f40ef5ad640 1 --2- 172.21.15.31:0/3954531814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e8100820 0x7f40e8102c50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.046+0000 7f40ef5ad640 1 -- 172.21.15.31:0/3954531814 >> 172.21.15.31:0/3954531814 conn(0x7f40e80fc440 msgr2=0x7f40e80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.047+0000 7f40ef5ad640 1 -- 172.21.15.31:0/3954531814 shutdown_connections 2024-04-25T13:17:34.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.047+0000 7f40ef5ad640 1 -- 172.21.15.31:0/3954531814 wait complete. 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.047+0000 7f40ef5ad640 1 Processor -- start 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.047+0000 7f40ef5ad640 1 -- start start 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.047+0000 7f40ef5ad640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40e8100820 0x7f40e819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.047+0000 7f40ef5ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e8103190 0x7f40e819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.047+0000 7f40ef5ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e8105b40 0x7f40e81a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.047+0000 7f40ef5ad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8114110 con 0x7f40e8105b40 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.047+0000 7f40ef5ad640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8113f90 con 0x7f40e8103190 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.047+0000 7f40ef5ad640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8114290 con 0x7f40e8100820 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40edb4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e8105b40 0x7f40e81a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40edb4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e8105b40 0x7f40e81a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36838/0 (socket says 172.21.15.31:36838) 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40edb4a640 1 -- 172.21.15.31:0/769430636 learned_addr learned my addr 172.21.15.31:0/769430636 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40ed349640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40e8100820 0x7f40e819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40ecb48640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e8103190 0x7f40e819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40edb4a640 1 -- 172.21.15.31:0/769430636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40e8100820 msgr2=0x7f40e819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:34.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40edb4a640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40e8100820 0x7f40e819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40edb4a640 1 -- 172.21.15.31:0/769430636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e8103190 msgr2=0x7f40e819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40edb4a640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e8103190 0x7f40e819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40edb4a640 1 -- 172.21.15.31:0/769430636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e81a63e0 con 0x7f40e8105b40 2024-04-25T13:17:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40ecb48640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e8103190 0x7f40e819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40edb4a640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e8105b40 0x7f40e81a5d70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f40e000e970 tx=0x7f40e000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40de7fc640 1 -- 172.21.15.31:0/769430636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e000ccc0 con 0x7f40e8105b40 2024-04-25T13:17:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40ef5ad640 1 -- 172.21.15.31:0/769430636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e81a66d0 con 0x7f40e8105b40 2024-04-25T13:17:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40de7fc640 1 -- 172.21.15.31:0/769430636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e000ce60 con 0x7f40e8105b40 2024-04-25T13:17:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40de7fc640 1 -- 172.21.15.31:0/769430636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0010430 con 0x7f40e8105b40 2024-04-25T13:17:34.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.048+0000 7f40ef5ad640 1 -- 172.21.15.31:0/769430636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e81a6c20 con 0x7f40e8105b40 2024-04-25T13:17:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.049+0000 7f40ef5ad640 1 -- 172.21.15.31:0/769430636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e80fdf30 con 0x7f40e8105b40 2024-04-25T13:17:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.050+0000 7f40de7fc640 1 -- 172.21.15.31:0/769430636 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f40e0004fb0 con 0x7f40e8105b40 2024-04-25T13:17:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.051+0000 7f40de7fc640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40d40a3bb0 0x7f40d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.051+0000 7f40ed349640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40d40a3bb0 0x7f40d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.051+0000 7f40de7fc640 1 -- 172.21.15.31:0/769430636 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f40e00d0600 con 0x7f40e8105b40 2024-04-25T13:17:34.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.052+0000 7f40ed349640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40d40a3bb0 0x7f40d40a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f40e4005e30 tx=0x7f40e4005f50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:34.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.055+0000 7f40de7fc640 1 -- 172.21.15.31:0/769430636 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40e0053ff0 con 0x7f40e8105b40 2024-04-25T13:17:34.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:34 smithi031 ceph-mon[29457]: Deploying daemon mds.b on smithi066 2024-04-25T13:17:34.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:34 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.31:6840/164386980,v1:172.21.15.31:6841/164386980] up:boot 2024-04-25T13:17:34.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:34 smithi031 ceph-mon[29457]: fsmap 11 up:standby 2024-04-25T13:17:34.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:34 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:17:34.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.282+0000 7f40ef5ad640 1 -- 172.21.15.31:0/769430636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40e8062b50 con 0x7f40e8105b40 2024-04-25T13:17:34.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.282+0000 7f40de7fc640 1 -- 172.21.15.31:0/769430636 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9647 (secure 0 0 0) 0x7f40e009d480 con 0x7f40e8105b40 2024-04-25T13:17:34.283 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:34.283 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2599371925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2599371925},{"type":"v1","addr":"172.21.15.31:6839","nonce":2599371925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/164386980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":164386980},{"type":"v1","addr":"172.21.15.31:6841","nonce":164386980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:17:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 12 2024-04-25T13:17:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.284+0000 7f40ef5ad640 1 -- 172.21.15.31:0/769430636 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40d40a3bb0 msgr2=0x7f40d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.285+0000 7f40ef5ad640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40d40a3bb0 0x7f40d40a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f40e4005e30 tx=0x7f40e4005f50 comp rx=0 tx=0).stop 2024-04-25T13:17:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.285+0000 7f40ef5ad640 1 -- 172.21.15.31:0/769430636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e8105b40 msgr2=0x7f40e81a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.285+0000 7f40ef5ad640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e8105b40 0x7f40e81a5d70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f40e000e970 tx=0x7f40e000ed70 comp rx=0 tx=0).stop 2024-04-25T13:17:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.285+0000 7f40ef5ad640 1 -- 172.21.15.31:0/769430636 shutdown_connections 2024-04-25T13:17:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.285+0000 7f40ef5ad640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f40d40a3bb0 0x7f40d40a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.285+0000 7f40ef5ad640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e8105b40 0x7f40e81a5d70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.285+0000 7f40ef5ad640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40e8103190 0x7f40e819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.285+0000 7f40ef5ad640 1 --2- 172.21.15.31:0/769430636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40e8100820 0x7f40e819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.285+0000 7f40ef5ad640 1 -- 172.21.15.31:0/769430636 >> 172.21.15.31:0/769430636 conn(0x7f40e80fc440 msgr2=0x7f40e8103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.285+0000 7f40ef5ad640 1 -- 172.21.15.31:0/769430636 shutdown_connections 2024-04-25T13:17:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:34.285+0000 7f40ef5ad640 1 -- 172.21.15.31:0/769430636 wait complete. 2024-04-25T13:17:34.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:34 smithi066 ceph-mon[33137]: Deploying daemon mds.b on smithi066 2024-04-25T13:17:34.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:34 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.31:6840/164386980,v1:172.21.15.31:6841/164386980] up:boot 2024-04-25T13:17:34.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:34 smithi066 ceph-mon[33137]: fsmap 11 up:standby 2024-04-25T13:17:34.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:34 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:17:34.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:34 smithi136 ceph-mon[33038]: Deploying daemon mds.b on smithi066 2024-04-25T13:17:34.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:34 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.31:6840/164386980,v1:172.21.15.31:6841/164386980] up:boot 2024-04-25T13:17:34.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:34 smithi136 ceph-mon[33038]: fsmap 11 up:standby 2024-04-25T13:17:34.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:34 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:17:35.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:35 smithi066 ceph-mon[33137]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:35.345 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:35 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/769430636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:35.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:35 smithi136 ceph-mon[33038]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:35.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:35 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/769430636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:35.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:35 smithi031 ceph-mon[29457]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:35.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:35 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/769430636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:35.637 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:17:35 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-b[63639]: starting mds.b at 2024-04-25T13:17:35.637 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:17:35 smithi066 bash[63624]: 3ff34147d10f4cb786c0adb36a72c48969cf712304a9396ccec46d3bf60c874f 2024-04-25T13:17:35.908 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:17:35 smithi066 systemd[1]: Started Ceph mds.b for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:17:36.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:36.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.401+0000 7f3433043640 1 -- 172.21.15.31:0/4187563338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c10a160 msgr2=0x7f342c1109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.401+0000 7f3433043640 1 --2- 172.21.15.31:0/4187563338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c10a160 0x7f342c1109f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3424009ca0 tx=0x7f342402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.401+0000 7f3433043640 1 -- 172.21.15.31:0/4187563338 shutdown_connections 2024-04-25T13:17:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.401+0000 7f3433043640 1 --2- 172.21.15.31:0/4187563338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c10a160 0x7f342c1109f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.401+0000 7f3433043640 1 --2- 172.21.15.31:0/4187563338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c1097a0 0x7f342c109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.401+0000 7f3433043640 1 --2- 172.21.15.31:0/4187563338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f342c1085b0 0x7f342c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.401+0000 7f3433043640 1 -- 172.21.15.31:0/4187563338 >> 172.21.15.31:0/4187563338 conn(0x7f342c077570 msgr2=0x7f342c079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.401+0000 7f3433043640 1 -- 172.21.15.31:0/4187563338 shutdown_connections 2024-04-25T13:17:36.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.401+0000 7f3433043640 1 -- 172.21.15.31:0/4187563338 wait complete. 2024-04-25T13:17:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.402+0000 7f3433043640 1 Processor -- start 2024-04-25T13:17:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.402+0000 7f3433043640 1 -- start start 2024-04-25T13:17:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f3433043640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c1085b0 0x7f342c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f3433043640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c1097a0 0x7f342c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f3433043640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f342c10a160 0x7f342c1a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f3433043640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1140e0 con 0x7f342c1097a0 2024-04-25T13:17:36.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f3433043640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f342c113f60 con 0x7f342c10a160 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f3433043640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f342c114260 con 0x7f342c1085b0 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f342bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c1097a0 0x7f342c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f342bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c1097a0 0x7f342c19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36860/0 (socket says 172.21.15.31:36860) 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f342bfff640 1 -- 172.21.15.31:0/4149714880 learned_addr learned my addr 172.21.15.31:0/4149714880 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f3430ddf640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c1085b0 0x7f342c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f34315e0640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f342c10a160 0x7f342c1a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f342bfff640 1 -- 172.21.15.31:0/4149714880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c1085b0 msgr2=0x7f342c19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f342bfff640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c1085b0 0x7f342c19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f342bfff640 1 -- 172.21.15.31:0/4149714880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f342c10a160 msgr2=0x7f342c1a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f342bfff640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f342c10a160 0x7f342c1a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f342bfff640 1 -- 172.21.15.31:0/4149714880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c1a63b0 con 0x7f342c1097a0 2024-04-25T13:17:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f342bfff640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c1097a0 0x7f342c19ecd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f341800e9f0 tx=0x7f341800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.403+0000 7f3429ffb640 1 -- 172.21.15.31:0/4149714880 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341800cd80 con 0x7f342c1097a0 2024-04-25T13:17:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.404+0000 7f3429ffb640 1 -- 172.21.15.31:0/4149714880 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34180054d0 con 0x7f342c1097a0 2024-04-25T13:17:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.404+0000 7f3433043640 1 -- 172.21.15.31:0/4149714880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c1a66a0 con 0x7f342c1097a0 2024-04-25T13:17:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.404+0000 7f3429ffb640 1 -- 172.21.15.31:0/4149714880 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418010690 con 0x7f342c1097a0 2024-04-25T13:17:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.404+0000 7f3433043640 1 -- 172.21.15.31:0/4149714880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1a6ba0 con 0x7f342c1097a0 2024-04-25T13:17:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.405+0000 7f3433043640 1 -- 172.21.15.31:0/4149714880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c10f8d0 con 0x7f342c1097a0 2024-04-25T13:17:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.406+0000 7f3429ffb640 1 -- 172.21.15.31:0/4149714880 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3418004e90 con 0x7f342c1097a0 2024-04-25T13:17:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.407+0000 7f3429ffb640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f341c0a3bb0 0x7f341c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.407+0000 7f3430ddf640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f341c0a3bb0 0x7f341c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.407+0000 7f3429ffb640 1 -- 172.21.15.31:0/4149714880 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f34180d0350 con 0x7f342c1097a0 2024-04-25T13:17:36.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.407+0000 7f3430ddf640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f341c0a3bb0 0x7f341c0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f342c109600 tx=0x7f342000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.412+0000 7f3429ffb640 1 -- 172.21.15.31:0/4149714880 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f341809d1d0 con 0x7f342c1097a0 2024-04-25T13:17:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.672+0000 7f3433043640 1 -- 172.21.15.31:0/4149714880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f342c02d5c0 con 0x7f342c1097a0 2024-04-25T13:17:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.673+0000 7f3429ffb640 1 -- 172.21.15.31:0/4149714880 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7f341809cad0 con 0x7f342c1097a0 2024-04-25T13:17:36.674 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:36.674 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2599371925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2599371925},{"type":"v1","addr":"172.21.15.31:6839","nonce":2599371925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/164386980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":164386980},{"type":"v1","addr":"172.21.15.31:6841","nonce":164386980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/2682191586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2682191586},{"type":"v1","addr":"172.21.15.66:6839","nonce":2682191586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:17:36.675 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 13 2024-04-25T13:17:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 -- 172.21.15.31:0/4149714880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f341c0a3bb0 msgr2=0x7f341c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f341c0a3bb0 0x7f341c0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f342c109600 tx=0x7f342000b390 comp rx=0 tx=0).stop 2024-04-25T13:17:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 -- 172.21.15.31:0/4149714880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c1097a0 msgr2=0x7f342c19ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c1097a0 0x7f342c19ecd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f341800e9f0 tx=0x7f341800edf0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 -- 172.21.15.31:0/4149714880 shutdown_connections 2024-04-25T13:17:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f341c0a3bb0 0x7f341c0a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f342c10a160 0x7f342c1a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c1097a0 0x7f342c19ecd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 --2- 172.21.15.31:0/4149714880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f342c1085b0 0x7f342c19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 -- 172.21.15.31:0/4149714880 >> 172.21.15.31:0/4149714880 conn(0x7f342c077570 msgr2=0x7f342c10caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 -- 172.21.15.31:0/4149714880 shutdown_connections 2024-04-25T13:17:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.675+0000 7f3433043640 1 -- 172.21.15.31:0/4149714880 wait complete. 2024-04-25T13:17:36.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:17:36.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.791+0000 7f7039660640 1 -- 172.21.15.31:0/427877769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7034103b70 msgr2=0x7f703410a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:36.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.791+0000 7f7039660640 1 --2- 172.21.15.31:0/427877769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7034103b70 0x7f703410a400 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f702c009ca0 tx=0x7f702c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.791+0000 7f7039660640 1 -- 172.21.15.31:0/427877769 shutdown_connections 2024-04-25T13:17:36.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.791+0000 7f7039660640 1 --2- 172.21.15.31:0/427877769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7034103b70 0x7f703410a400 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.791+0000 7f7039660640 1 --2- 172.21.15.31:0/427877769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70341031b0 0x7f7034103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.791+0000 7f7039660640 1 --2- 172.21.15.31:0/427877769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7034101fc0 0x7f70341023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.791+0000 7f7039660640 1 -- 172.21.15.31:0/427877769 >> 172.21.15.31:0/427877769 conn(0x7f70340fd530 msgr2=0x7f70340ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:36.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.791+0000 7f7039660640 1 -- 172.21.15.31:0/427877769 shutdown_connections 2024-04-25T13:17:36.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.791+0000 7f7039660640 1 -- 172.21.15.31:0/427877769 wait complete. 2024-04-25T13:17:36.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.791+0000 7f7039660640 1 Processor -- start 2024-04-25T13:17:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f7039660640 1 -- start start 2024-04-25T13:17:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f7039660640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7034101fc0 0x7f70341981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f7039660640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70341031b0 0x7f70341986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f7039660640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7034103b70 0x7f703419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:36.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f7039660640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f703410db00 con 0x7f70341031b0 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f7039660640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f703410d980 con 0x7f7034103b70 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f7039660640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f703410dc80 con 0x7f7034101fc0 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f7032ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7034101fc0 0x7f70341981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f70337fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7034103b70 0x7f703419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f7032ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7034101fc0 0x7f70341981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.31:58596/0 (socket says 172.21.15.31:58596) 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f7032ffd640 1 -- 172.21.15.31:0/1513737818 learned_addr learned my addr 172.21.15.31:0/1513737818 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.792+0000 7f70327fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70341031b0 0x7f70341986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f7032ffd640 1 -- 172.21.15.31:0/1513737818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7034103b70 msgr2=0x7f703419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f7032ffd640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7034103b70 0x7f703419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f7032ffd640 1 -- 172.21.15.31:0/1513737818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70341031b0 msgr2=0x7f70341986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f7032ffd640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70341031b0 0x7f70341986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f7032ffd640 1 -- 172.21.15.31:0/1513737818 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7034074200 con 0x7f7034101fc0 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f70327fc640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70341031b0 0x7f70341986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:17:36.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f70337fe640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7034103b70 0x7f703419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:17:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f7032ffd640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7034101fc0 0x7f70341981a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f702400e9c0 tx=0x7f702400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f701bfff640 1 -- 172.21.15.31:0/1513737818 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702400cde0 con 0x7f7034101fc0 2024-04-25T13:17:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f701bfff640 1 -- 172.21.15.31:0/1513737818 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70240108e0 con 0x7f7034101fc0 2024-04-25T13:17:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f701bfff640 1 -- 172.21.15.31:0/1513737818 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024018960 con 0x7f7034101fc0 2024-04-25T13:17:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f7039660640 1 -- 172.21.15.31:0/1513737818 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70340744f0 con 0x7f7034101fc0 2024-04-25T13:17:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.793+0000 7f7039660640 1 -- 172.21.15.31:0/1513737818 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7034074a40 con 0x7f7034101fc0 2024-04-25T13:17:36.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.795+0000 7f7039660640 1 -- 172.21.15.31:0/1513737818 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7034075e20 con 0x7f7034101fc0 2024-04-25T13:17:36.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.795+0000 7f701bfff640 1 -- 172.21.15.31:0/1513737818 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f702401f020 con 0x7f7034101fc0 2024-04-25T13:17:36.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.796+0000 7f701bfff640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f701c0a3c30 0x7f701c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:36.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.796+0000 7f70327fc640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f701c0a3c30 0x7f701c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:36.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.796+0000 7f701bfff640 1 -- 172.21.15.31:0/1513737818 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f70240d0280 con 0x7f7034101fc0 2024-04-25T13:17:36.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.796+0000 7f70327fc640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f701c0a3c30 0x7f701c0a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70341996b0 tx=0x7f7028005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:36.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.801+0000 7f701bfff640 1 -- 172.21.15.31:0/1513737818 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7024053ff0 con 0x7f7034101fc0 2024-04-25T13:17:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:36 smithi136 ceph-mon[33038]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:36 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.66:6838/2682191586,v1:172.21.15.66:6839/2682191586] up:boot 2024-04-25T13:17:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:36 smithi136 ceph-mon[33038]: fsmap 12 up:standby 2024-04-25T13:17:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:36 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:17:36.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:36 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/4149714880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:36 smithi031 ceph-mon[29457]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:36 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.66:6838/2682191586,v1:172.21.15.66:6839/2682191586] up:boot 2024-04-25T13:17:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:36 smithi031 ceph-mon[29457]: fsmap 12 up:standby 2024-04-25T13:17:36.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:36 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:17:36.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:36 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/4149714880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:36.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:36.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:36 smithi066 ceph-mon[33137]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:36.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:36.998 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:36 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.66:6838/2682191586,v1:172.21.15.66:6839/2682191586] up:boot 2024-04-25T13:17:36.999 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:36 smithi066 ceph-mon[33137]: fsmap 12 up:standby 2024-04-25T13:17:36.999 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:36 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:17:36.999 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:36 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/4149714880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:37.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.997+0000 7f7039660640 1 -- 172.21.15.31:0/1513737818 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7034108e90 con 0x7f7034101fc0 2024-04-25T13:17:37.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:36.998+0000 7f701bfff640 1 -- 172.21.15.31:0/1513737818 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18419 (secure 0 0 0) 0x7f702409d060 con 0x7f7034101fc0 2024-04-25T13:17:37.000 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:37.000 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":84,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:16:53.053945+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:17:37.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.000+0000 7f7039660640 1 -- 172.21.15.31:0/1513737818 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f701c0a3c30 msgr2=0x7f701c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.000+0000 7f7039660640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f701c0a3c30 0x7f701c0a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70341996b0 tx=0x7f7028005f90 comp rx=0 tx=0).stop 2024-04-25T13:17:37.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.000+0000 7f7039660640 1 -- 172.21.15.31:0/1513737818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7034101fc0 msgr2=0x7f70341981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.000+0000 7f7039660640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7034101fc0 0x7f70341981a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f702400e9c0 tx=0x7f702400ee80 comp rx=0 tx=0).stop 2024-04-25T13:17:37.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.000+0000 7f7039660640 1 -- 172.21.15.31:0/1513737818 shutdown_connections 2024-04-25T13:17:37.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.000+0000 7f7039660640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f701c0a3c30 0x7f701c0a60f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.000+0000 7f7039660640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7034103b70 0x7f703419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.000+0000 7f7039660640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70341031b0 0x7f70341986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.000+0000 7f7039660640 1 --2- 172.21.15.31:0/1513737818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7034101fc0 0x7f70341981a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.000+0000 7f7039660640 1 -- 172.21.15.31:0/1513737818 >> 172.21.15.31:0/1513737818 conn(0x7f70340fd530 msgr2=0x7f70341064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:37.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.001+0000 7f7039660640 1 -- 172.21.15.31:0/1513737818 shutdown_connections 2024-04-25T13:17:37.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.001+0000 7f7039660640 1 -- 172.21.15.31:0/1513737818 wait complete. 2024-04-25T13:17:37.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:13:01.793580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:17:37.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:17:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.118+0000 7fb0100d0640 1 -- 172.21.15.31:0/1274548409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb008105360 msgr2=0x7fb008107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.118+0000 7fb0100d0640 1 --2- 172.21.15.31:0/1274548409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb008105360 0x7fb008107750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faff8009900 tx=0x7faff802a410 comp rx=0 tx=0).stop 2024-04-25T13:17:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.118+0000 7fb0100d0640 1 -- 172.21.15.31:0/1274548409 shutdown_connections 2024-04-25T13:17:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.118+0000 7fb0100d0640 1 --2- 172.21.15.31:0/1274548409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb008107c90 0x7fb00810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.118+0000 7fb0100d0640 1 --2- 172.21.15.31:0/1274548409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb008105360 0x7fb008107750 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.118+0000 7fb0100d0640 1 --2- 172.21.15.31:0/1274548409 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb008102a30 0x7fb008104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.118+0000 7fb0100d0640 1 -- 172.21.15.31:0/1274548409 >> 172.21.15.31:0/1274548409 conn(0x7fb0080fc440 msgr2=0x7fb0080fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.118+0000 7fb0100d0640 1 -- 172.21.15.31:0/1274548409 shutdown_connections 2024-04-25T13:17:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.118+0000 7fb0100d0640 1 -- 172.21.15.31:0/1274548409 wait complete. 2024-04-25T13:17:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.118+0000 7fb0100d0640 1 Processor -- start 2024-04-25T13:17:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.118+0000 7fb0100d0640 1 -- start start 2024-04-25T13:17:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb0100d0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb008102a30 0x7fb008198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb0100d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb008105360 0x7fb0081987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb0100d0640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb008107c90 0x7fb00819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb0100d0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb00810daf0 con 0x7fb008105360 2024-04-25T13:17:37.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb0100d0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb00810d970 con 0x7fb008102a30 2024-04-25T13:17:37.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb0100d0640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb00810dc70 con 0x7fb008107c90 2024-04-25T13:17:37.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb00d66b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb008105360 0x7fb0081987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb00d66b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb008105360 0x7fb0081987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36890/0 (socket says 172.21.15.31:36890) 2024-04-25T13:17:37.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb00d66b640 1 -- 172.21.15.31:0/3606824994 learned_addr learned my addr 172.21.15.31:0/3606824994 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:37.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb00de6c640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb008102a30 0x7fb008198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb00e66d640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb008107c90 0x7fb00819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb00d66b640 1 -- 172.21.15.31:0/3606824994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb008107c90 msgr2=0x7fb00819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb00d66b640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb008107c90 0x7fb00819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb00d66b640 1 -- 172.21.15.31:0/3606824994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb008102a30 msgr2=0x7fb008198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb00d66b640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb008102a30 0x7fb008198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb00d66b640 1 -- 172.21.15.31:0/3606824994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00819fe40 con 0x7fb008105360 2024-04-25T13:17:37.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb00d66b640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb008105360 0x7fb0081987b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faff8006010 tx=0x7faff80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:37.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7faffeffd640 1 -- 172.21.15.31:0/3606824994 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff803f030 con 0x7fb008105360 2024-04-25T13:17:37.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7faffeffd640 1 -- 172.21.15.31:0/3606824994 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff80318f0 con 0x7fb008105360 2024-04-25T13:17:37.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb0100d0640 1 -- 172.21.15.31:0/3606824994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0081a00d0 con 0x7fb008105360 2024-04-25T13:17:37.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7faffeffd640 1 -- 172.21.15.31:0/3606824994 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8035070 con 0x7fb008105360 2024-04-25T13:17:37.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.119+0000 7fb0100d0640 1 -- 172.21.15.31:0/3606824994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0081a05c0 con 0x7fb008105360 2024-04-25T13:17:37.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.120+0000 7fb0100d0640 1 -- 172.21.15.31:0/3606824994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0080fdf30 con 0x7fb008105360 2024-04-25T13:17:37.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.122+0000 7faffeffd640 1 -- 172.21.15.31:0/3606824994 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faff8042030 con 0x7fb008105360 2024-04-25T13:17:37.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.122+0000 7faffeffd640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faff40a3bb0 0x7faff40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.122+0000 7fb00de6c640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faff40a3bb0 0x7faff40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.122+0000 7faffeffd640 1 -- 172.21.15.31:0/3606824994 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7faff80f0800 con 0x7fb008105360 2024-04-25T13:17:37.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.123+0000 7fb00de6c640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faff40a3bb0 0x7faff40a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb004005fd0 tx=0x7fb004009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:37.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.126+0000 7faffeffd640 1 -- 172.21.15.31:0/3606824994 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faff80bd030 con 0x7fb008105360 2024-04-25T13:17:37.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.324+0000 7fb0100d0640 1 -- 172.21.15.31:0/3606824994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb00802d810 con 0x7fb008105360 2024-04-25T13:17:37.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.324+0000 7faffeffd640 1 -- 172.21.15.31:0/3606824994 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7faff80c1330 con 0x7fb008105360 2024-04-25T13:17:37.325 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T13:17:37.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 -- 172.21.15.31:0/3606824994 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faff40a3bb0 msgr2=0x7faff40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faff40a3bb0 0x7faff40a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb004005fd0 tx=0x7fb004009680 comp rx=0 tx=0).stop 2024-04-25T13:17:37.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 -- 172.21.15.31:0/3606824994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb008105360 msgr2=0x7fb0081987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb008105360 0x7fb0081987b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faff8006010 tx=0x7faff80042f0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 -- 172.21.15.31:0/3606824994 shutdown_connections 2024-04-25T13:17:37.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7faff40a3bb0 0x7faff40a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb008107c90 0x7fb00819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb008105360 0x7fb0081987b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 --2- 172.21.15.31:0/3606824994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb008102a30 0x7fb008198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 -- 172.21.15.31:0/3606824994 >> 172.21.15.31:0/3606824994 conn(0x7fb0080fc440 msgr2=0x7fb0081034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:37.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 -- 172.21.15.31:0/3606824994 shutdown_connections 2024-04-25T13:17:37.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.326+0000 7fb0100d0640 1 -- 172.21.15.31:0/3606824994 wait complete. 2024-04-25T13:17:37.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:17:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.447+0000 7f11a4e2c640 1 -- 172.21.15.31:0/3782928566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0100e80 msgr2=0x7f11a01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.447+0000 7f11a4e2c640 1 --2- 172.21.15.31:0/3782928566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0100e80 0x7f11a01012a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1190009900 tx=0x7f119002a410 comp rx=0 tx=0).stop 2024-04-25T13:17:37.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.447+0000 7f11a4e2c640 1 -- 172.21.15.31:0/3782928566 shutdown_connections 2024-04-25T13:17:37.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.447+0000 7f11a4e2c640 1 --2- 172.21.15.31:0/3782928566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a0102a20 0x7f11a0109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.447+0000 7f11a4e2c640 1 --2- 172.21.15.31:0/3782928566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a0102060 0x7f11a01024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.447+0000 7f11a4e2c640 1 --2- 172.21.15.31:0/3782928566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0100e80 0x7f11a01012a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.447+0000 7f11a4e2c640 1 -- 172.21.15.31:0/3782928566 >> 172.21.15.31:0/3782928566 conn(0x7f11a00fc440 msgr2=0x7f11a00fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:37.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.447+0000 7f11a4e2c640 1 -- 172.21.15.31:0/3782928566 shutdown_connections 2024-04-25T13:17:37.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.447+0000 7f11a4e2c640 1 -- 172.21.15.31:0/3782928566 wait complete. 2024-04-25T13:17:37.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.447+0000 7f11a4e2c640 1 Processor -- start 2024-04-25T13:17:37.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f11a4e2c640 1 -- start start 2024-04-25T13:17:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f11a4e2c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a0100e80 0x7f11a0197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f11a4e2c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a0102060 0x7f11a0197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f11a4e2c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0102a20 0x7f11a019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f11a4e2c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f11a010ca20 con 0x7f11a0102a20 2024-04-25T13:17:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f11a4e2c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11a010c8a0 con 0x7f11a0100e80 2024-04-25T13:17:37.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f11a4e2c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11a010cba0 con 0x7f11a0102060 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0102a20 0x7f11a019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119dd9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a0102060 0x7f11a0197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0102a20 0x7f11a019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36910/0 (socket says 172.21.15.31:36910) 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119ed9d640 1 -- 172.21.15.31:0/1234715596 learned_addr learned my addr 172.21.15.31:0/1234715596 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a0100e80 0x7f11a0197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119e59c640 1 -- 172.21.15.31:0/1234715596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a0102060 msgr2=0x7f11a0197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119e59c640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a0102060 0x7f11a0197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119e59c640 1 -- 172.21.15.31:0/1234715596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0102a20 msgr2=0x7f11a019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119e59c640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0102a20 0x7f11a019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119e59c640 1 -- 172.21.15.31:0/1234715596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a019ed70 con 0x7f11a0100e80 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119ed9d640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0102a20 0x7f11a019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.448+0000 7f119dd9b640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a0102060 0x7f11a0197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.449+0000 7f119e59c640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a0100e80 0x7f11a0197150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1190006010 tx=0x7f11900042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:37.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.449+0000 7f118f7fe640 1 -- 172.21.15.31:0/1234715596 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119003f030 con 0x7f11a0100e80 2024-04-25T13:17:37.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.449+0000 7f11a4e2c640 1 -- 172.21.15.31:0/1234715596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a019f000 con 0x7f11a0100e80 2024-04-25T13:17:37.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.449+0000 7f118f7fe640 1 -- 172.21.15.31:0/1234715596 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1190031910 con 0x7f11a0100e80 2024-04-25T13:17:37.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.449+0000 7f118f7fe640 1 -- 172.21.15.31:0/1234715596 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190035070 con 0x7f11a0100e80 2024-04-25T13:17:37.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.449+0000 7f11a4e2c640 1 -- 172.21.15.31:0/1234715596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a019f4f0 con 0x7f11a0100e80 2024-04-25T13:17:37.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.450+0000 7f11a4e2c640 1 -- 172.21.15.31:0/1234715596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a01082f0 con 0x7f11a0100e80 2024-04-25T13:17:37.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.451+0000 7f118f7fe640 1 -- 172.21.15.31:0/1234715596 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1190042030 con 0x7f11a0100e80 2024-04-25T13:17:37.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.452+0000 7f118f7fe640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f11880a3bb0 0x7f11880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.452+0000 7f119dd9b640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f11880a3bb0 0x7f11880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.452+0000 7f118f7fe640 1 -- 172.21.15.31:0/1234715596 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f11900f0680 con 0x7f11a0100e80 2024-04-25T13:17:37.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.452+0000 7f119dd9b640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f11880a3bb0 0x7f11880a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f11a0198660 tx=0x7f1194005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:37.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.456+0000 7f118f7fe640 1 -- 172.21.15.31:0/1234715596 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11900bcf50 con 0x7f11a0100e80 2024-04-25T13:17:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.663+0000 7f11a4e2c640 1 -- 172.21.15.31:0/1234715596 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f11a002d790 con 0x7f11a0100e80 2024-04-25T13:17:37.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.665+0000 7f118f7fe640 1 -- 172.21.15.31:0/1234715596 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7f11900c1250 con 0x7f11a0100e80 2024-04-25T13:17:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:37.666 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-04-25T13:17:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:17:37.666 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-04-25T13:17:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-04-25T13:17:37.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:17:37.667 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-04-25T13:17:37.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.667+0000 7f11a4e2c640 1 -- 172.21.15.31:0/1234715596 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f11880a3bb0 msgr2=0x7f11880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.667+0000 7f11a4e2c640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f11880a3bb0 0x7f11880a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f11a0198660 tx=0x7f1194005f90 comp rx=0 tx=0).stop 2024-04-25T13:17:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.667+0000 7f11a4e2c640 1 -- 172.21.15.31:0/1234715596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a0100e80 msgr2=0x7f11a0197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.667+0000 7f11a4e2c640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a0100e80 0x7f11a0197150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1190006010 tx=0x7f11900042f0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.667+0000 7f11a4e2c640 1 -- 172.21.15.31:0/1234715596 shutdown_connections 2024-04-25T13:17:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.667+0000 7f11a4e2c640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f11880a3bb0 0x7f11880a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.667+0000 7f11a4e2c640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a0102a20 0x7f11a019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.667+0000 7f11a4e2c640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a0102060 0x7f11a0197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.667+0000 7f11a4e2c640 1 --2- 172.21.15.31:0/1234715596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11a0100e80 0x7f11a0197150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.667+0000 7f11a4e2c640 1 -- 172.21.15.31:0/1234715596 >> 172.21.15.31:0/1234715596 conn(0x7f11a00fc440 msgr2=0x7f11a01054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.668+0000 7f11a4e2c640 1 -- 172.21.15.31:0/1234715596 shutdown_connections 2024-04-25T13:17:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.668+0000 7f11a4e2c640 1 -- 172.21.15.31:0/1234715596 wait complete. 2024-04-25T13:17:37.681 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:17:37.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:17:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.787+0000 7f06ad8f5640 1 -- 172.21.15.31:0/395872414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06a8100250 msgr2=0x7f06a8100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.787+0000 7f06ad8f5640 1 --2- 172.21.15.31:0/395872414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06a8100250 0x7f06a8100670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f069c009930 tx=0x7f069c02a420 comp rx=0 tx=0).stop 2024-04-25T13:17:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.787+0000 7f06ad8f5640 1 -- 172.21.15.31:0/395872414 shutdown_connections 2024-04-25T13:17:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.787+0000 7f06ad8f5640 1 --2- 172.21.15.31:0/395872414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8103be0 0x7f06a810a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.787+0000 7f06ad8f5640 1 --2- 172.21.15.31:0/395872414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a8103220 0x7f06a81036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.787+0000 7f06ad8f5640 1 --2- 172.21.15.31:0/395872414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06a8100250 0x7f06a8100670 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.787+0000 7f06ad8f5640 1 -- 172.21.15.31:0/395872414 >> 172.21.15.31:0/395872414 conn(0x7f06a80fbb80 msgr2=0x7f06a80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.787+0000 7f06ad8f5640 1 -- 172.21.15.31:0/395872414 shutdown_connections 2024-04-25T13:17:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.787+0000 7f06ad8f5640 1 -- 172.21.15.31:0/395872414 wait complete. 2024-04-25T13:17:37.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06ad8f5640 1 Processor -- start 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06ad8f5640 1 -- start start 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06ad8f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06a8100250 0x7f06a8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06ad8f5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8103220 0x7f06a81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06ad8f5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a8103be0 0x7f06a819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06ad8f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f06a810db40 con 0x7f06a8100250 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06ad8f5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f06a810d9c0 con 0x7f06a8103220 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06ad8f5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06a810dcc0 con 0x7f06a8103be0 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06a8100250 0x7f06a8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06a6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06a8100250 0x7f06a8198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36918/0 (socket says 172.21.15.31:36918) 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06a6ffd640 1 -- 172.21.15.31:0/751879289 learned_addr learned my addr 172.21.15.31:0/751879289 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06a67fc640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8103220 0x7f06a81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06a77fe640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a8103be0 0x7f06a819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06a6ffd640 1 -- 172.21.15.31:0/751879289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a8103be0 msgr2=0x7f06a819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06a6ffd640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a8103be0 0x7f06a819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06a6ffd640 1 -- 172.21.15.31:0/751879289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8103220 msgr2=0x7f06a81987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06a6ffd640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8103220 0x7f06a81987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06a6ffd640 1 -- 172.21.15.31:0/751879289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a819fe90 con 0x7f06a8100250 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.788+0000 7f06a77fe640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a8103be0 0x7f06a819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.789+0000 7f06a6ffd640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06a8100250 0x7f06a8198270 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f069c031500 tx=0x7f069c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.789+0000 7f06ac8f3640 1 -- 172.21.15.31:0/751879289 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c03f070 con 0x7f06a8100250 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.789+0000 7f06ac8f3640 1 -- 172.21.15.31:0/751879289 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f069c031c50 con 0x7f06a8100250 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.789+0000 7f06ac8f3640 1 -- 172.21.15.31:0/751879289 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c035030 con 0x7f06a8100250 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.789+0000 7f06ad8f5640 1 -- 172.21.15.31:0/751879289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a81a0120 con 0x7f06a8100250 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.789+0000 7f06ad8f5640 1 -- 172.21.15.31:0/751879289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a81a05c0 con 0x7f06a8100250 2024-04-25T13:17:37.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.790+0000 7f06ad8f5640 1 -- 172.21.15.31:0/751879289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a8109410 con 0x7f06a8100250 2024-04-25T13:17:37.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.791+0000 7f06ac8f3640 1 -- 172.21.15.31:0/751879289 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f069c042070 con 0x7f06a8100250 2024-04-25T13:17:37.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.792+0000 7f06ac8f3640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06900a3bb0 0x7f06900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:37.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.792+0000 7f06ac8f3640 1 -- 172.21.15.31:0/751879289 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f069c0f0740 con 0x7f06a8100250 2024-04-25T13:17:37.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.792+0000 7f06a67fc640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06900a3bb0 0x7f06900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:37.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.792+0000 7f06a67fc640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06900a3bb0 0x7f06900a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f06a8199780 tx=0x7f069800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:37.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.796+0000 7f06ac8f3640 1 -- 172.21.15.31:0/751879289 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f069c0bd5c0 con 0x7f06a8100250 2024-04-25T13:17:37.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:37.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1513737818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:37.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3606824994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:17:37.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:37.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:37.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:37.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:37.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1234715596' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:17:37.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:37.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:37.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:37.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:37.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:37 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:37.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:37.988+0000 7f06ad8f5640 1 -- 172.21.15.31:0/751879289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f06a802d5c0 con 0x7f06a8100250 2024-04-25T13:17:38.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1513737818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:38.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3606824994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:17:38.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1234715596' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:17:38.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:38.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:38.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:37 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1513737818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:38.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3606824994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:17:38.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1234715596' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:17:38.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:38.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:38.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:37 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.786+0000 7f06ac8f3640 1 -- 172.21.15.31:0/751879289 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7f069c0bcec0 con 0x7f06a8100250 2024-04-25T13:17:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-04-25T13:17:38.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.788+0000 7f06ad8f5640 1 -- 172.21.15.31:0/751879289 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06900a3bb0 msgr2=0x7f06900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.788+0000 7f06ad8f5640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06900a3bb0 0x7f06900a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f06a8199780 tx=0x7f069800b390 comp rx=0 tx=0).stop 2024-04-25T13:17:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.788+0000 7f06ad8f5640 1 -- 172.21.15.31:0/751879289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06a8100250 msgr2=0x7f06a8198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.788+0000 7f06ad8f5640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06a8100250 0x7f06a8198270 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f069c031500 tx=0x7f069c003c10 comp rx=0 tx=0).stop 2024-04-25T13:17:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.788+0000 7f06ad8f5640 1 -- 172.21.15.31:0/751879289 shutdown_connections 2024-04-25T13:17:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.788+0000 7f06ad8f5640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06900a3bb0 0x7f06900a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.788+0000 7f06ad8f5640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06a8103be0 0x7f06a819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.788+0000 7f06ad8f5640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06a8103220 0x7f06a81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.788+0000 7f06ad8f5640 1 --2- 172.21.15.31:0/751879289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06a8100250 0x7f06a8198270 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.788+0000 7f06ad8f5640 1 -- 172.21.15.31:0/751879289 >> 172.21.15.31:0/751879289 conn(0x7f06a80fbb80 msgr2=0x7f06a81065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.789+0000 7f06ad8f5640 1 -- 172.21.15.31:0/751879289 shutdown_connections 2024-04-25T13:17:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.789+0000 7f06ad8f5640 1 -- 172.21.15.31:0/751879289 wait complete. 2024-04-25T13:17:38.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:17:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.913+0000 7f483954a640 1 -- 172.21.15.31:0/1035594037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834105340 msgr2=0x7f4834107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.913+0000 7f483954a640 1 --2- 172.21.15.31:0/1035594037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834105340 0x7f4834107730 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4828009900 tx=0x7f482802a410 comp rx=0 tx=0).stop 2024-04-25T13:17:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.913+0000 7f483954a640 1 -- 172.21.15.31:0/1035594037 shutdown_connections 2024-04-25T13:17:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.913+0000 7f483954a640 1 --2- 172.21.15.31:0/1035594037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4834107c70 0x7f483410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.913+0000 7f483954a640 1 --2- 172.21.15.31:0/1035594037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834105340 0x7f4834107730 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.913+0000 7f483954a640 1 --2- 172.21.15.31:0/1035594037 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4834102a10 0x7f4834104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.913+0000 7f483954a640 1 -- 172.21.15.31:0/1035594037 >> 172.21.15.31:0/1035594037 conn(0x7f48340fc400 msgr2=0x7f48340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.913+0000 7f483954a640 1 -- 172.21.15.31:0/1035594037 shutdown_connections 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.913+0000 7f483954a640 1 -- 172.21.15.31:0/1035594037 wait complete. 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f483954a640 1 Processor -- start 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f483954a640 1 -- start start 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f483954a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4834102a10 0x7f4834198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f483954a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834105340 0x7f4834198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f483954a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4834107c70 0x7f483419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f483954a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f483410dab0 con 0x7f4834105340 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f483954a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f483410d930 con 0x7f4834102a10 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f483954a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f483410dc30 con 0x7f4834107c70 2024-04-25T13:17:38.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f48327fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834105340 0x7f4834198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f48327fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834105340 0x7f4834198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36944/0 (socket says 172.21.15.31:36944) 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f48327fc640 1 -- 172.21.15.31:0/2555918424 learned_addr learned my addr 172.21.15.31:0/2555918424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f4832ffd640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4834102a10 0x7f4834198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.914+0000 7f48337fe640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4834107c70 0x7f483419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.915+0000 7f48327fc640 1 -- 172.21.15.31:0/2555918424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4834107c70 msgr2=0x7f483419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.915+0000 7f48327fc640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4834107c70 0x7f483419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.915+0000 7f48327fc640 1 -- 172.21.15.31:0/2555918424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4834102a10 msgr2=0x7f4834198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.915+0000 7f48327fc640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4834102a10 0x7f4834198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.915+0000 7f48327fc640 1 -- 172.21.15.31:0/2555918424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483419fe50 con 0x7f4834105340 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.915+0000 7f48327fc640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834105340 0x7f4834198770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4828006010 tx=0x7f48280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.915+0000 7f481bfff640 1 -- 172.21.15.31:0/2555918424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482803f030 con 0x7f4834105340 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.915+0000 7f481bfff640 1 -- 172.21.15.31:0/2555918424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4828031910 con 0x7f4834105340 2024-04-25T13:17:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.915+0000 7f481bfff640 1 -- 172.21.15.31:0/2555918424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828035070 con 0x7f4834105340 2024-04-25T13:17:38.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.915+0000 7f483954a640 1 -- 172.21.15.31:0/2555918424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48341a00e0 con 0x7f4834105340 2024-04-25T13:17:38.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.915+0000 7f483954a640 1 -- 172.21.15.31:0/2555918424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48341a05d0 con 0x7f4834105340 2024-04-25T13:17:38.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.916+0000 7f483954a640 1 -- 172.21.15.31:0/2555918424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48340fdef0 con 0x7f4834105340 2024-04-25T13:17:38.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.917+0000 7f481bfff640 1 -- 172.21.15.31:0/2555918424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4828042030 con 0x7f4834105340 2024-04-25T13:17:38.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.918+0000 7f481bfff640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f481c0a3bb0 0x7f481c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:38.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.918+0000 7f4832ffd640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f481c0a3bb0 0x7f481c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:38.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.918+0000 7f481bfff640 1 -- 172.21.15.31:0/2555918424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f48280f0a30 con 0x7f4834105340 2024-04-25T13:17:38.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.918+0000 7f4832ffd640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f481c0a3bb0 0x7f481c0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4824009bd0 tx=0x7f4824005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:38.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:38.922+0000 7f481bfff640 1 -- 172.21.15.31:0/2555918424 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4828003e60 con 0x7f4834105340 2024-04-25T13:17:38.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:38 smithi031 ceph-mon[29457]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:38.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:38 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:38.943 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:38 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/751879289' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:17:38.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:38 smithi136 ceph-mon[33038]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:38.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:38 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:38.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:38 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/751879289' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:17:39.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.111+0000 7f483954a640 1 -- 172.21.15.31:0/2555918424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f483402d5c0 con 0x7f4834105340 2024-04-25T13:17:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:38 smithi066 ceph-mon[33137]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:38 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:38 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/751879289' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:17:39.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.795+0000 7f481bfff640 1 -- 172.21.15.31:0/2555918424 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7f48280bd050 con 0x7f4834105340 2024-04-25T13:17:39.797 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-04-25T13:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.797+0000 7f483954a640 1 -- 172.21.15.31:0/2555918424 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f481c0a3bb0 msgr2=0x7f481c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.797+0000 7f483954a640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f481c0a3bb0 0x7f481c0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4824009bd0 tx=0x7f4824005f90 comp rx=0 tx=0).stop 2024-04-25T13:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.797+0000 7f483954a640 1 -- 172.21.15.31:0/2555918424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834105340 msgr2=0x7f4834198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.797+0000 7f483954a640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834105340 0x7f4834198770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4828006010 tx=0x7f48280042f0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.798+0000 7f483954a640 1 -- 172.21.15.31:0/2555918424 shutdown_connections 2024-04-25T13:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.798+0000 7f483954a640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f481c0a3bb0 0x7f481c0a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.798+0000 7f483954a640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4834107c70 0x7f483419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.798+0000 7f483954a640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834105340 0x7f4834198770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.798+0000 7f483954a640 1 --2- 172.21.15.31:0/2555918424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4834102a10 0x7f4834198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.798+0000 7f483954a640 1 -- 172.21.15.31:0/2555918424 >> 172.21.15.31:0/2555918424 conn(0x7f48340fc400 msgr2=0x7f48341048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.798+0000 7f483954a640 1 -- 172.21.15.31:0/2555918424 shutdown_connections 2024-04-25T13:17:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.798+0000 7f483954a640 1 -- 172.21.15.31:0/2555918424 wait complete. 2024-04-25T13:17:39.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:17:39.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.923+0000 7fc85b64e640 1 -- 172.21.15.31:0/3446615083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc854102a10 msgr2=0x7fc854104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:39.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.923+0000 7fc85b64e640 1 --2- 172.21.15.31:0/3446615083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc854102a10 0x7fc854104e00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc850009900 tx=0x7fc85002a410 comp rx=0 tx=0).stop 2024-04-25T13:17:39.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.923+0000 7fc85b64e640 1 -- 172.21.15.31:0/3446615083 shutdown_connections 2024-04-25T13:17:39.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.923+0000 7fc85b64e640 1 --2- 172.21.15.31:0/3446615083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc854107c70 0x7fc85410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.923+0000 7fc85b64e640 1 --2- 172.21.15.31:0/3446615083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854105340 0x7fc854107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.923+0000 7fc85b64e640 1 --2- 172.21.15.31:0/3446615083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc854102a10 0x7fc854104e00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.923+0000 7fc85b64e640 1 -- 172.21.15.31:0/3446615083 >> 172.21.15.31:0/3446615083 conn(0x7fc8540fc400 msgr2=0x7fc8540fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.923+0000 7fc85b64e640 1 -- 172.21.15.31:0/3446615083 shutdown_connections 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.923+0000 7fc85b64e640 1 -- 172.21.15.31:0/3446615083 wait complete. 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.923+0000 7fc85b64e640 1 Processor -- start 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc85b64e640 1 -- start start 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc85b64e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc854102a10 0x7fc854198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc85b64e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc854105340 0x7fc854198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc85b64e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854107c70 0x7fc85419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc85b64e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc85410dad0 con 0x7fc854105340 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc85b64e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc85410d950 con 0x7fc854102a10 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc85b64e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc85410dc50 con 0x7fc854107c70 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc858be9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc854105340 0x7fc854198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc858be9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc854105340 0x7fc854198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36966/0 (socket says 172.21.15.31:36966) 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc858be9640 1 -- 172.21.15.31:0/12155240 learned_addr learned my addr 172.21.15.31:0/12155240 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:39.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc859beb640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854107c70 0x7fc85419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc8593ea640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc854102a10 0x7fc854198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc858be9640 1 -- 172.21.15.31:0/12155240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854107c70 msgr2=0x7fc85419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc858be9640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854107c70 0x7fc85419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc858be9640 1 -- 172.21.15.31:0/12155240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc854102a10 msgr2=0x7fc854198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc858be9640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc854102a10 0x7fc854198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc858be9640 1 -- 172.21.15.31:0/12155240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85419fe70 con 0x7fc854105340 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc8593ea640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc854102a10 0x7fc854198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc858be9640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc854105340 0x7fc854198790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc84400e9f0 tx=0x7fc84400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc84a7fc640 1 -- 172.21.15.31:0/12155240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84400ccc0 con 0x7fc854105340 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc84a7fc640 1 -- 172.21.15.31:0/12155240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8440054d0 con 0x7fc854105340 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc85b64e640 1 -- 172.21.15.31:0/12155240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8541a0160 con 0x7fc854105340 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.924+0000 7fc84a7fc640 1 -- 172.21.15.31:0/12155240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844010430 con 0x7fc854105340 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.925+0000 7fc85b64e640 1 -- 172.21.15.31:0/12155240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8541a06b0 con 0x7fc854105340 2024-04-25T13:17:39.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.925+0000 7fc85b64e640 1 -- 172.21.15.31:0/12155240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8540fdef0 con 0x7fc854105340 2024-04-25T13:17:39.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.927+0000 7fc84a7fc640 1 -- 172.21.15.31:0/12155240 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc8440046c0 con 0x7fc854105340 2024-04-25T13:17:39.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.928+0000 7fc84a7fc640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc8400a3c30 0x7fc8400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:39.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.928+0000 7fc8593ea640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc8400a3c30 0x7fc8400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:39.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.928+0000 7fc84a7fc640 1 -- 172.21.15.31:0/12155240 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7879+0+0 (secure 0 0 0) 0x7fc8440d07f0 con 0x7fc854105340 2024-04-25T13:17:39.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.929+0000 7fc8593ea640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc8400a3c30 0x7fc8400a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc850006010 tx=0x7fc850000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:39.932+0000 7fc84a7fc640 1 -- 172.21.15.31:0/12155240 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc844053ff0 con 0x7fc854105340 2024-04-25T13:17:40.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/751879289' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:17:40.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:17:40.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2555918424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:17:40.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:40.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:40.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.160+0000 7fc85b64e640 1 -- 172.21.15.31:0/12155240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc85402d5c0 con 0x7fc854105340 2024-04-25T13:17:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/751879289' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:17:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:17:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2555918424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:17:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:40.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/751879289' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2555918424' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:17:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.797+0000 7fc84a7fc640 1 -- 172.21.15.31:0/12155240 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc84409d380 con 0x7fc854105340 2024-04-25T13:17:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T13:17:40.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.801+0000 7fc85b64e640 1 -- 172.21.15.31:0/12155240 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc8400a3c30 msgr2=0x7fc8400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:40.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.801+0000 7fc85b64e640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc8400a3c30 0x7fc8400a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc850006010 tx=0x7fc850000bc0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.801+0000 7fc85b64e640 1 -- 172.21.15.31:0/12155240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc854105340 msgr2=0x7fc854198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:40.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.801+0000 7fc85b64e640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc854105340 0x7fc854198790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc84400e9f0 tx=0x7fc84400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.802+0000 7fc85b64e640 1 -- 172.21.15.31:0/12155240 shutdown_connections 2024-04-25T13:17:40.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.802+0000 7fc85b64e640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc8400a3c30 0x7fc8400a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.802+0000 7fc85b64e640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854107c70 0x7fc85419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.802+0000 7fc85b64e640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc854105340 0x7fc854198790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.802+0000 7fc85b64e640 1 --2- 172.21.15.31:0/12155240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc854102a10 0x7fc854198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.802+0000 7fc85b64e640 1 -- 172.21.15.31:0/12155240 >> 172.21.15.31:0/12155240 conn(0x7fc8540fc400 msgr2=0x7fc8541048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:40.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.802+0000 7fc85b64e640 1 -- 172.21.15.31:0/12155240 shutdown_connections 2024-04-25T13:17:40.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.802+0000 7fc85b64e640 1 -- 172.21.15.31:0/12155240 wait complete. 2024-04-25T13:17:40.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:17:40.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/660086648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b64101fc0 msgr2=0x7f7b641023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 --2- 172.21.15.31:0/660086648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b64101fc0 0x7f7b641023e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009ca0 tx=0x7f7b5c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/660086648 shutdown_connections 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 --2- 172.21.15.31:0/660086648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b64103b70 0x7f7b6410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 --2- 172.21.15.31:0/660086648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b641031b0 0x7f7b64103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 --2- 172.21.15.31:0/660086648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b64101fc0 0x7f7b641023e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/660086648 >> 172.21.15.31:0/660086648 conn(0x7f7b640fd550 msgr2=0x7f7b640ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/660086648 shutdown_connections 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/660086648 wait complete. 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 Processor -- start 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 -- start start 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b64101fc0 0x7f7b6419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.936+0000 7f7b6b8a9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b641031b0 0x7f7b6419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:40.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b6b8a9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b64103b70 0x7f7b641a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b6b8a9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6410db00 con 0x7f7b641031b0 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b6b8a9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6410d980 con 0x7f7b64103b70 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b6b8a9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6410dc80 con 0x7f7b64101fc0 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b68e44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b641031b0 0x7f7b6419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b68e44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b641031b0 0x7f7b6419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36984/0 (socket says 172.21.15.31:36984) 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b68e44640 1 -- 172.21.15.31:0/3924459467 learned_addr learned my addr 172.21.15.31:0/3924459467 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b69645640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b64101fc0 0x7f7b6419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b69e46640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b64103b70 0x7f7b641a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b68e44640 1 -- 172.21.15.31:0/3924459467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b64101fc0 msgr2=0x7f7b6419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b68e44640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b64101fc0 0x7f7b6419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b68e44640 1 -- 172.21.15.31:0/3924459467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b64103b70 msgr2=0x7f7b641a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b68e44640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b64103b70 0x7f7b641a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:40.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b68e44640 1 -- 172.21.15.31:0/3924459467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b641a2080 con 0x7f7b641031b0 2024-04-25T13:17:40.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b68e44640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b641031b0 0x7f7b6419a9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800d950 tx=0x7f7b5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:40.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b627fc640 1 -- 172.21.15.31:0/3924459467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58017070 con 0x7f7b641031b0 2024-04-25T13:17:40.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b627fc640 1 -- 172.21.15.31:0/3924459467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b58009c30 con 0x7f7b641031b0 2024-04-25T13:17:40.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/3924459467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b641a2370 con 0x7f7b641031b0 2024-04-25T13:17:40.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b627fc640 1 -- 172.21.15.31:0/3924459467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5800f620 con 0x7f7b641031b0 2024-04-25T13:17:40.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.937+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/3924459467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b641a2910 con 0x7f7b641031b0 2024-04-25T13:17:40.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.938+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/3924459467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b641a2b70 con 0x7f7b641031b0 2024-04-25T13:17:40.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.939+0000 7f7b627fc640 1 -- 172.21.15.31:0/3924459467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7b58010460 con 0x7f7b641031b0 2024-04-25T13:17:40.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.940+0000 7f7b627fc640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b4c0a3c30 0x7f7b4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:40.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.940+0000 7f7b69645640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b4c0a3c30 0x7f7b4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:40.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.940+0000 7f7b627fc640 1 -- 172.21.15.31:0/3924459467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f7b580cfe50 con 0x7f7b641031b0 2024-04-25T13:17:40.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.941+0000 7f7b69645640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b4c0a3c30 0x7f7b4c0a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000f80 tx=0x7f7b5c005dc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:40.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:40.945+0000 7f7b627fc640 1 -- 172.21.15.31:0/3924459467 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b58009de0 con 0x7f7b641031b0 2024-04-25T13:17:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:40 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2555918424' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:17:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:40 smithi136 ceph-mon[33038]: pgmap v289: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:40 smithi136 ceph-mon[33038]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:17:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:40 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/12155240' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:17:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:40 smithi136 ceph-mon[33038]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T13:17:41.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.132+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/3924459467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b641a2d40 con 0x7f7b641031b0 2024-04-25T13:17:41.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.134+0000 7f7b627fc640 1 -- 172.21.15.31:0/3924459467 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22195 (secure 0 0 0) 0x7f7b5809c290 con 0x7f7b641031b0 2024-04-25T13:17:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:41.135 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:17:40.162708+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:17:41.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/3924459467 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b4c0a3c30 msgr2=0x7f7b4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:41.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b4c0a3c30 0x7f7b4c0a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000f80 tx=0x7f7b5c005dc0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/3924459467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b641031b0 msgr2=0x7f7b6419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:41.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b641031b0 0x7f7b6419a9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800d950 tx=0x7f7b5800de10 comp rx=0 tx=0).stop 2024-04-25T13:17:41.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/3924459467 shutdown_connections 2024-04-25T13:17:41.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f7b4c0a3c30 0x7f7b4c0a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b64103b70 0x7f7b641a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b641031b0 0x7f7b6419a9f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 --2- 172.21.15.31:0/3924459467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b64101fc0 0x7f7b6419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/3924459467 >> 172.21.15.31:0/3924459467 conn(0x7f7b640fd550 msgr2=0x7f7b64074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:41.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/3924459467 shutdown_connections 2024-04-25T13:17:41.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.136+0000 7f7b6b8a9640 1 -- 172.21.15.31:0/3924459467 wait complete. 2024-04-25T13:17:41.146 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:40 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2555918424' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:17:41.146 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:40 smithi031 ceph-mon[29457]: pgmap v289: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:41.147 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:40 smithi031 ceph-mon[29457]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:17:41.147 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:40 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/12155240' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:17:41.147 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:40 smithi031 ceph-mon[29457]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T13:17:41.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:17:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:40 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2555918424' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:17:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:40 smithi066 ceph-mon[33137]: pgmap v289: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:17:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:40 smithi066 ceph-mon[33137]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:17:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:40 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/12155240' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:17:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:40 smithi066 ceph-mon[33137]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T13:17:41.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 -- 172.21.15.31:0/2989553532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc134100e60 msgr2=0x7fc134101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:41.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 --2- 172.21.15.31:0/2989553532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc134100e60 0x7fc134101280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc124009900 tx=0x7fc12402a410 comp rx=0 tx=0).stop 2024-04-25T13:17:41.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 -- 172.21.15.31:0/2989553532 shutdown_connections 2024-04-25T13:17:41.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 --2- 172.21.15.31:0/2989553532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134102a00 0x7fc134109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 --2- 172.21.15.31:0/2989553532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc134102040 0x7fc1341024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 --2- 172.21.15.31:0/2989553532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc134100e60 0x7fc134101280 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 -- 172.21.15.31:0/2989553532 >> 172.21.15.31:0/2989553532 conn(0x7fc1340fc440 msgr2=0x7fc1340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 -- 172.21.15.31:0/2989553532 shutdown_connections 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 -- 172.21.15.31:0/2989553532 wait complete. 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 Processor -- start 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 -- start start 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc134100e60 0x7fc13419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134102040 0x7fc13419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc134102a00 0x7fc1341a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc13410ca40 con 0x7fc134100e60 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc13410c8c0 con 0x7fc134102a00 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.250+0000 7fc13b136640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc13410cbc0 con 0x7fc134102040 2024-04-25T13:17:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc138ed2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc134100e60 0x7fc13419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc138ed2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc134100e60 0x7fc13419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36998/0 (socket says 172.21.15.31:36998) 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc138ed2640 1 -- 172.21.15.31:0/403050621 learned_addr learned my addr 172.21.15.31:0/403050621 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc133fff640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134102040 0x7fc13419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc138ed2640 1 -- 172.21.15.31:0/403050621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134102040 msgr2=0x7fc13419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc138ed2640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134102040 0x7fc13419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc138ed2640 1 -- 172.21.15.31:0/403050621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc134102a00 msgr2=0x7fc1341a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc1396d3640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc134102a00 0x7fc1341a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc138ed2640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc134102a00 0x7fc1341a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc138ed2640 1 -- 172.21.15.31:0/403050621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1341a2050 con 0x7fc134100e60 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc138ed2640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc134100e60 0x7fc13419a430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc124031500 tx=0x7fc124003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc131ffb640 1 -- 172.21.15.31:0/403050621 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12403f070 con 0x7fc134100e60 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc131ffb640 1 -- 172.21.15.31:0/403050621 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc124003e70 con 0x7fc134100e60 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc13b136640 1 -- 172.21.15.31:0/403050621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1341a22e0 con 0x7fc134100e60 2024-04-25T13:17:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc131ffb640 1 -- 172.21.15.31:0/403050621 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124035030 con 0x7fc134100e60 2024-04-25T13:17:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.251+0000 7fc13b136640 1 -- 172.21.15.31:0/403050621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1341a2820 con 0x7fc134100e60 2024-04-25T13:17:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.252+0000 7fc13b136640 1 -- 172.21.15.31:0/403050621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1341a2b40 con 0x7fc134100e60 2024-04-25T13:17:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.253+0000 7fc131ffb640 1 -- 172.21.15.31:0/403050621 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc124042070 con 0x7fc134100e60 2024-04-25T13:17:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.254+0000 7fc131ffb640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc11c0a3c30 0x7fc11c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.254+0000 7fc133fff640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc11c0a3c30 0x7fc11c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:41.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.254+0000 7fc131ffb640 1 -- 172.21.15.31:0/403050621 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fc1240f0fc0 con 0x7fc134100e60 2024-04-25T13:17:41.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.254+0000 7fc133fff640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc11c0a3c30 0x7fc11c0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc128005fd0 tx=0x7fc128005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:41.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.259+0000 7fc131ffb640 1 -- 172.21.15.31:0/403050621 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc1240bda50 con 0x7fc134100e60 2024-04-25T13:17:41.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.450+0000 7fc13b136640 1 -- 172.21.15.31:0/403050621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1341a2d10 con 0x7fc134100e60 2024-04-25T13:17:41.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.451+0000 7fc131ffb640 1 -- 172.21.15.31:0/403050621 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22195 (secure 0 0 0) 0x7fc1240bd350 con 0x7fc134100e60 2024-04-25T13:17:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:41.453 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:17:40.162708+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:17:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 -- 172.21.15.31:0/403050621 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc11c0a3c30 msgr2=0x7fc11c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc11c0a3c30 0x7fc11c0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc128005fd0 tx=0x7fc128005850 comp rx=0 tx=0).stop 2024-04-25T13:17:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 -- 172.21.15.31:0/403050621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc134100e60 msgr2=0x7fc13419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc134100e60 0x7fc13419a430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc124031500 tx=0x7fc124003c80 comp rx=0 tx=0).stop 2024-04-25T13:17:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 -- 172.21.15.31:0/403050621 shutdown_connections 2024-04-25T13:17:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc11c0a3c30 0x7fc11c0a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc134102a00 0x7fc1341a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc134102040 0x7fc13419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 --2- 172.21.15.31:0/403050621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc134100e60 0x7fc13419a430 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 -- 172.21.15.31:0/403050621 >> 172.21.15.31:0/403050621 conn(0x7fc1340fc440 msgr2=0x7fc1340fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:41.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 -- 172.21.15.31:0/403050621 shutdown_connections 2024-04-25T13:17:41.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.454+0000 7fc13b136640 1 -- 172.21.15.31:0/403050621 wait complete. 2024-04-25T13:17:41.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:17:41.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.571+0000 7f06ede31640 1 -- 172.21.15.31:0/3293040184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06e8101fe0 msgr2=0x7f06e8102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:41.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.571+0000 7f06ede31640 1 --2- 172.21.15.31:0/3293040184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06e8101fe0 0x7f06e8102460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009900 tx=0x7f06d802a410 comp rx=0 tx=0).stop 2024-04-25T13:17:41.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.571+0000 7f06ede31640 1 -- 172.21.15.31:0/3293040184 shutdown_connections 2024-04-25T13:17:41.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.571+0000 7f06ede31640 1 --2- 172.21.15.31:0/3293040184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06e81029a0 0x7f06e81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.571+0000 7f06ede31640 1 --2- 172.21.15.31:0/3293040184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06e8101fe0 0x7f06e8102460 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.571+0000 7f06ede31640 1 --2- 172.21.15.31:0/3293040184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8100e90 0x7f06e81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.571+0000 7f06ede31640 1 -- 172.21.15.31:0/3293040184 >> 172.21.15.31:0/3293040184 conn(0x7f06e80fc400 msgr2=0x7f06e80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:41.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.571+0000 7f06ede31640 1 -- 172.21.15.31:0/3293040184 shutdown_connections 2024-04-25T13:17:41.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.571+0000 7f06ede31640 1 -- 172.21.15.31:0/3293040184 wait complete. 2024-04-25T13:17:41.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.571+0000 7f06ede31640 1 Processor -- start 2024-04-25T13:17:41.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.571+0000 7f06ede31640 1 -- start start 2024-04-25T13:17:41.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06ede31640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06e8100e90 0x7f06e8198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:41.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06ede31640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8101fe0 0x7f06e8198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:41.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06ede31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06e81029a0 0x7f06e819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:41.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06ede31640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f06e810ca40 con 0x7f06e81029a0 2024-04-25T13:17:41.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06ede31640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f06e810c8c0 con 0x7f06e8100e90 2024-04-25T13:17:41.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06ede31640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06e810cbc0 con 0x7f06e8101fe0 2024-04-25T13:17:41.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06e81029a0 0x7f06e819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:41.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06e81029a0 0x7f06e819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37028/0 (socket says 172.21.15.31:37028) 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e7fff640 1 -- 172.21.15.31:0/1929367325 learned_addr learned my addr 172.21.15.31:0/1929367325 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e77fe640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06e8100e90 0x7f06e8198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e6ffd640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8101fe0 0x7f06e8198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e7fff640 1 -- 172.21.15.31:0/1929367325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8101fe0 msgr2=0x7f06e8198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e7fff640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8101fe0 0x7f06e8198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e7fff640 1 -- 172.21.15.31:0/1929367325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06e8100e90 msgr2=0x7f06e8198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e7fff640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06e8100e90 0x7f06e8198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e7fff640 1 -- 172.21.15.31:0/1929367325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e819fe60 con 0x7f06e81029a0 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e77fe640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06e8100e90 0x7f06e8198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.572+0000 7f06e7fff640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06e81029a0 0x7f06e819f7f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f06e000eaa0 tx=0x7f06e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.573+0000 7f06e4ff9640 1 -- 172.21.15.31:0/1929367325 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e000cdb0 con 0x7f06e81029a0 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.573+0000 7f06e4ff9640 1 -- 172.21.15.31:0/1929367325 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e0005020 con 0x7f06e81029a0 2024-04-25T13:17:41.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.573+0000 7f06ede31640 1 -- 172.21.15.31:0/1929367325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e81a0150 con 0x7f06e81029a0 2024-04-25T13:17:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.573+0000 7f06e4ff9640 1 -- 172.21.15.31:0/1929367325 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e00116d0 con 0x7f06e81029a0 2024-04-25T13:17:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.573+0000 7f06ede31640 1 -- 172.21.15.31:0/1929367325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e81a0620 con 0x7f06e81029a0 2024-04-25T13:17:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.574+0000 7f06ede31640 1 -- 172.21.15.31:0/1929367325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e81a08e0 con 0x7f06e81029a0 2024-04-25T13:17:41.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.575+0000 7f06e4ff9640 1 -- 172.21.15.31:0/1929367325 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f06e00051d0 con 0x7f06e81029a0 2024-04-25T13:17:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.575+0000 7f06e4ff9640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06d00a3bb0 0x7f06d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.576+0000 7f06e4ff9640 1 -- 172.21.15.31:0/1929367325 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f06e00d0810 con 0x7f06e81029a0 2024-04-25T13:17:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.576+0000 7f06e77fe640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06d00a3bb0 0x7f06d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:41.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.576+0000 7f06e77fe640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06d00a3bb0 0x7f06d00a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009fc0 tx=0x7f06dc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:41.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.583+0000 7f06e4ff9640 1 -- 172.21.15.31:0/1929367325 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06e0016030 con 0x7f06e81029a0 2024-04-25T13:17:41.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:41.812+0000 7f06ede31640 1 -- 172.21.15.31:0/1929367325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f06e81a0c30 con 0x7f06e81029a0 2024-04-25T13:17:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:41 smithi066 ceph-mon[33137]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:17:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:41 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/12155240' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:17:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:41 smithi066 ceph-mon[33137]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:17:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:41 smithi066 ceph-mon[33137]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T13:17:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:41 smithi066 ceph-mon[33137]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T13:17:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:41 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3924459467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:41 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/403050621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:41 smithi136 ceph-mon[33038]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:17:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:41 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/12155240' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:17:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:41 smithi136 ceph-mon[33038]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:17:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:41 smithi136 ceph-mon[33038]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T13:17:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:41 smithi136 ceph-mon[33038]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T13:17:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:41 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3924459467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:41 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/403050621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:42.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:41 smithi031 ceph-mon[29457]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:17:42.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:41 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/12155240' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:17:42.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:41 smithi031 ceph-mon[29457]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:17:42.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:41 smithi031 ceph-mon[29457]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T13:17:42.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:41 smithi031 ceph-mon[29457]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T13:17:42.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:41 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3924459467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:42.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:41 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/403050621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:42.816 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:17:42 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i[63643]: 2024-04-25T13:17:42.531+0000 7fd5d0289700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:17:42.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.823+0000 7f06e4ff9640 1 -- 172.21.15.31:0/1929367325 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f06e009ccd0 con 0x7f06e81029a0 2024-04-25T13:17:42.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 -- 172.21.15.31:0/1929367325 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06d00a3bb0 msgr2=0x7f06d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:42.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06d00a3bb0 0x7f06d00a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009fc0 tx=0x7f06dc005f90 comp rx=0 tx=0).stop 2024-04-25T13:17:42.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 -- 172.21.15.31:0/1929367325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06e81029a0 msgr2=0x7f06e819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06e81029a0 0x7f06e819f7f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f06e000eaa0 tx=0x7f06e000ef60 comp rx=0 tx=0).stop 2024-04-25T13:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 -- 172.21.15.31:0/1929367325 shutdown_connections 2024-04-25T13:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f06d00a3bb0 0x7f06d00a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06e81029a0 0x7f06e819f7f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8101fe0 0x7f06e8198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 --2- 172.21.15.31:0/1929367325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f06e8100e90 0x7f06e8198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 -- 172.21.15.31:0/1929367325 >> 172.21.15.31:0/1929367325 conn(0x7f06e80fc400 msgr2=0x7f06e80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 -- 172.21.15.31:0/1929367325 shutdown_connections 2024-04-25T13:17:42.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.826+0000 7f06ede31640 1 -- 172.21.15.31:0/1929367325 wait complete. 2024-04-25T13:17:42.841 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T13:17:42.841 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.952+0000 7fa0c6289640 1 -- 172.21.15.31:0/1202551404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c0103b50 msgr2=0x7fa0c010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.952+0000 7fa0c6289640 1 --2- 172.21.15.31:0/1202551404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c0103b50 0x7fa0c010a3e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009ca0 tx=0x7fa0b802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.952+0000 7fa0c6289640 1 -- 172.21.15.31:0/1202551404 shutdown_connections 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.952+0000 7fa0c6289640 1 --2- 172.21.15.31:0/1202551404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c0103b50 0x7fa0c010a3e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.952+0000 7fa0c6289640 1 --2- 172.21.15.31:0/1202551404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c0103190 0x7fa0c0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.952+0000 7fa0c6289640 1 --2- 172.21.15.31:0/1202551404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0c0101fa0 0x7fa0c01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.952+0000 7fa0c6289640 1 -- 172.21.15.31:0/1202551404 >> 172.21.15.31:0/1202551404 conn(0x7fa0c00fd530 msgr2=0x7fa0c00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.952+0000 7fa0c6289640 1 -- 172.21.15.31:0/1202551404 shutdown_connections 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.952+0000 7fa0c6289640 1 -- 172.21.15.31:0/1202551404 wait complete. 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0c6289640 1 Processor -- start 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0c6289640 1 -- start start 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0c6289640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0c0101fa0 0x7fa0c019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:42.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0c6289640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c0103190 0x7fa0c019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0c6289640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c0103b50 0x7fa0c01a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0c6289640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010dbc0 con 0x7fa0c0103190 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0c6289640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010da40 con 0x7fa0c0103b50 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0c6289640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010dd40 con 0x7fa0c0101fa0 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c0103190 0x7fa0c019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0bf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c0103190 0x7fa0c019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50732/0 (socket says 172.21.15.31:50732) 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0bf7fe640 1 -- 172.21.15.31:0/542541087 learned_addr learned my addr 172.21.15.31:0/542541087 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0bffff640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0c0101fa0 0x7fa0c019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0c4826640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c0103b50 0x7fa0c01a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0bf7fe640 1 -- 172.21.15.31:0/542541087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0c0101fa0 msgr2=0x7fa0c019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0bf7fe640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0c0101fa0 0x7fa0c019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0bf7fe640 1 -- 172.21.15.31:0/542541087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c0103b50 msgr2=0x7fa0c01a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0bf7fe640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c0103b50 0x7fa0c01a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.953+0000 7fa0bf7fe640 1 -- 172.21.15.31:0/542541087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c01a2130 con 0x7fa0c0103190 2024-04-25T13:17:42.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.954+0000 7fa0bffff640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0c0101fa0 0x7fa0c019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:17:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.954+0000 7fa0bf7fe640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c0103190 0x7fa0c019aa50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400b4a0 tx=0x7fa0b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.954+0000 7fa0bd7fa640 1 -- 172.21.15.31:0/542541087 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4014070 con 0x7fa0c0103190 2024-04-25T13:17:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.954+0000 7fa0bd7fa640 1 -- 172.21.15.31:0/542541087 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b4010ac0 con 0x7fa0c0103190 2024-04-25T13:17:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.954+0000 7fa0c6289640 1 -- 172.21.15.31:0/542541087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c01a2420 con 0x7fa0c0103190 2024-04-25T13:17:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.954+0000 7fa0bd7fa640 1 -- 172.21.15.31:0/542541087 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4017c30 con 0x7fa0c0103190 2024-04-25T13:17:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.954+0000 7fa0c6289640 1 -- 172.21.15.31:0/542541087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c01a29f0 con 0x7fa0c0103190 2024-04-25T13:17:42.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.955+0000 7fa0c6289640 1 -- 172.21.15.31:0/542541087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c01a2cb0 con 0x7fa0c0103190 2024-04-25T13:17:42.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.956+0000 7fa0bd7fa640 1 -- 172.21.15.31:0/542541087 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa0b4017070 con 0x7fa0c0103190 2024-04-25T13:17:42.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.957+0000 7fa0bd7fa640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa0a80a3bb0 0x7fa0a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:42.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.957+0000 7fa0bffff640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa0a80a3bb0 0x7fa0a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:42.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.957+0000 7fa0bd7fa640 1 -- 172.21.15.31:0/542541087 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa0b40cfae0 con 0x7fa0c0103190 2024-04-25T13:17:42.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.957+0000 7fa0bffff640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa0a80a3bb0 0x7fa0a80a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0102ff0 tx=0x7fa0b0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:42.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:42.962+0000 7fa0bd7fa640 1 -- 172.21.15.31:0/542541087 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0b409c570 con 0x7fa0c0103190 2024-04-25T13:17:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:42 smithi066 ceph-mon[33137]: pgmap v292: 97 pgs: 11 creating+peering, 26 active+clean, 60 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-25T13:17:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:42 smithi066 ceph-mon[33137]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:17:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:42 smithi066 ceph-mon[33137]: Cluster is now healthy 2024-04-25T13:17:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:42 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] up:active 2024-04-25T13:17:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:42 smithi066 ceph-mon[33137]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:42 smithi066 ceph-mon[33137]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:17:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:42 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1929367325' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:17:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:42 smithi066 ceph-mon[33137]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.189+0000 7fa0c6289640 1 -- 172.21.15.31:0/542541087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fa0c01a30a0 con 0x7fa0c0103190 2024-04-25T13:17:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:42 smithi136 ceph-mon[33038]: pgmap v292: 97 pgs: 11 creating+peering, 26 active+clean, 60 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-25T13:17:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:42 smithi136 ceph-mon[33038]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:17:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:42 smithi136 ceph-mon[33038]: Cluster is now healthy 2024-04-25T13:17:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:42 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] up:active 2024-04-25T13:17:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:42 smithi136 ceph-mon[33038]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:42 smithi136 ceph-mon[33038]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:17:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:42 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1929367325' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:17:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:42 smithi136 ceph-mon[33038]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:42 smithi031 ceph-mon[29457]: pgmap v292: 97 pgs: 11 creating+peering, 26 active+clean, 60 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-25T13:17:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:42 smithi031 ceph-mon[29457]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:17:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:42 smithi031 ceph-mon[29457]: Cluster is now healthy 2024-04-25T13:17:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:42 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] up:active 2024-04-25T13:17:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:42 smithi031 ceph-mon[29457]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:42 smithi031 ceph-mon[29457]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:17:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:42 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1929367325' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:17:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:42 smithi031 ceph-mon[29457]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:43.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.829+0000 7fa0bd7fa640 1 -- 172.21.15.31:0/542541087 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fa0c01a30a0 con 0x7fa0c0103190 2024-04-25T13:17:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 -- 172.21.15.31:0/542541087 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa0a80a3bb0 msgr2=0x7fa0a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa0a80a3bb0 0x7fa0a80a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0102ff0 tx=0x7fa0b0008040 comp rx=0 tx=0).stop 2024-04-25T13:17:43.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 -- 172.21.15.31:0/542541087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c0103190 msgr2=0x7fa0c019aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c0103190 0x7fa0c019aa50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400b4a0 tx=0x7fa0b400b960 comp rx=0 tx=0).stop 2024-04-25T13:17:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 -- 172.21.15.31:0/542541087 shutdown_connections 2024-04-25T13:17:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa0a80a3bb0 0x7fa0a80a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0c0103b50 0x7fa0c01a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c0103190 0x7fa0c019aa50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 --2- 172.21.15.31:0/542541087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0c0101fa0 0x7fa0c019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 -- 172.21.15.31:0/542541087 >> 172.21.15.31:0/542541087 conn(0x7fa0c00fd530 msgr2=0x7fa0c0108980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 -- 172.21.15.31:0/542541087 shutdown_connections 2024-04-25T13:17:43.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.832+0000 7fa0c6289640 1 -- 172.21.15.31:0/542541087 wait complete. 2024-04-25T13:17:43.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T13:17:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.958+0000 7f5b19587640 1 -- 172.21.15.31:0/2583820143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b141053e0 msgr2=0x7f5b141077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.958+0000 7f5b19587640 1 --2- 172.21.15.31:0/2583820143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b141053e0 0x7f5b141077d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009900 tx=0x7f5b0402a410 comp rx=0 tx=0).stop 2024-04-25T13:17:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.958+0000 7f5b19587640 1 -- 172.21.15.31:0/2583820143 shutdown_connections 2024-04-25T13:17:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.958+0000 7f5b19587640 1 --2- 172.21.15.31:0/2583820143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b14107d10 0x7f5b1410a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.958+0000 7f5b19587640 1 --2- 172.21.15.31:0/2583820143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b141053e0 0x7f5b141077d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.958+0000 7f5b19587640 1 --2- 172.21.15.31:0/2583820143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b14102ab0 0x7f5b14104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.958+0000 7f5b19587640 1 -- 172.21.15.31:0/2583820143 >> 172.21.15.31:0/2583820143 conn(0x7f5b140fc420 msgr2=0x7f5b140fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.958+0000 7f5b19587640 1 -- 172.21.15.31:0/2583820143 shutdown_connections 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.958+0000 7f5b19587640 1 -- 172.21.15.31:0/2583820143 wait complete. 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b19587640 1 Processor -- start 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b19587640 1 -- start start 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b19587640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b14102ab0 0x7f5b14198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b19587640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b141053e0 0x7f5b141987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b19587640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14107d10 0x7f5b1419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b19587640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1410daf0 con 0x7f5b14107d10 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b19587640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1410d970 con 0x7f5b14102ab0 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b19587640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1410dc70 con 0x7f5b141053e0 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14107d10 0x7f5b1419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14107d10 0x7f5b1419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50742/0 (socket says 172.21.15.31:50742) 2024-04-25T13:17:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b137fe640 1 -- 172.21.15.31:0/2387786846 learned_addr learned my addr 172.21.15.31:0/2387786846 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b12ffd640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b14102ab0 0x7f5b14198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.959+0000 7f5b127fc640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b141053e0 0x7f5b141987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.960+0000 7f5b137fe640 1 -- 172.21.15.31:0/2387786846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b141053e0 msgr2=0x7f5b141987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.960+0000 7f5b137fe640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b141053e0 0x7f5b141987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.960+0000 7f5b137fe640 1 -- 172.21.15.31:0/2387786846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b14102ab0 msgr2=0x7f5b14198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.960+0000 7f5b137fe640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b14102ab0 0x7f5b14198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.960+0000 7f5b137fe640 1 -- 172.21.15.31:0/2387786846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1419fe90 con 0x7f5b14107d10 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.960+0000 7f5b137fe640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14107d10 0x7f5b1419f820 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009fd0 tx=0x7f5b0c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.960+0000 7f5afbfff640 1 -- 172.21.15.31:0/2387786846 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c017070 con 0x7f5b14107d10 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.960+0000 7f5b19587640 1 -- 172.21.15.31:0/2387786846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b141a0180 con 0x7f5b14107d10 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.960+0000 7f5afbfff640 1 -- 172.21.15.31:0/2387786846 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b0c003e20 con 0x7f5b14107d10 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.960+0000 7f5afbfff640 1 -- 172.21.15.31:0/2387786846 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c010460 con 0x7f5b14107d10 2024-04-25T13:17:43.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.960+0000 7f5b19587640 1 -- 172.21.15.31:0/2387786846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b141a06a0 con 0x7f5b14107d10 2024-04-25T13:17:43.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.961+0000 7f5b19587640 1 -- 172.21.15.31:0/2387786846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b140fdf10 con 0x7f5b14107d10 2024-04-25T13:17:43.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.962+0000 7f5afbfff640 1 -- 172.21.15.31:0/2387786846 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b0c003970 con 0x7f5b14107d10 2024-04-25T13:17:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.963+0000 7f5afbfff640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5afc0a3c30 0x7f5afc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.963+0000 7f5b12ffd640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5afc0a3c30 0x7f5afc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.963+0000 7f5afbfff640 1 -- 172.21.15.31:0/2387786846 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5b0c0d0130 con 0x7f5b14107d10 2024-04-25T13:17:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.963+0000 7f5b12ffd640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5afc0a3c30 0x7f5afc0a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005fd0 tx=0x7f5b08005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:43.966+0000 7f5afbfff640 1 -- 172.21.15.31:0/2387786846 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b0c09cbc0 con 0x7f5b14107d10 2024-04-25T13:17:44.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:43 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1929367325' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:17:44.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:43 smithi066 ceph-mon[33137]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:44.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:43 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/542541087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:17:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.193+0000 7f5b19587640 1 -- 172.21.15.31:0/2387786846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5b1402d5c0 con 0x7f5b14107d10 2024-04-25T13:17:44.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:43 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1929367325' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:17:44.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:43 smithi136 ceph-mon[33038]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:44.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:43 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/542541087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:17:44.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:43 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1929367325' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:17:44.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:43 smithi031 ceph-mon[29457]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:44.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:43 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/542541087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:17:44.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.837+0000 7f5afbfff640 1 -- 172.21.15.31:0/2387786846 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f5b0c09c4c0 con 0x7f5b14107d10 2024-04-25T13:17:44.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.841+0000 7f5b19587640 1 -- 172.21.15.31:0/2387786846 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5afc0a3c30 msgr2=0x7f5afc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.841+0000 7f5b19587640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5afc0a3c30 0x7f5afc0a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005fd0 tx=0x7f5b08005850 comp rx=0 tx=0).stop 2024-04-25T13:17:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.841+0000 7f5b19587640 1 -- 172.21.15.31:0/2387786846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14107d10 msgr2=0x7f5b1419f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.841+0000 7f5b19587640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14107d10 0x7f5b1419f820 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009fd0 tx=0x7f5b0c00b260 comp rx=0 tx=0).stop 2024-04-25T13:17:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.841+0000 7f5b19587640 1 -- 172.21.15.31:0/2387786846 shutdown_connections 2024-04-25T13:17:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.841+0000 7f5b19587640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5afc0a3c30 0x7f5afc0a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.841+0000 7f5b19587640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14107d10 0x7f5b1419f820 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.841+0000 7f5b19587640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b141053e0 0x7f5b141987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.841+0000 7f5b19587640 1 --2- 172.21.15.31:0/2387786846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b14102ab0 0x7f5b14198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.841+0000 7f5b19587640 1 -- 172.21.15.31:0/2387786846 >> 172.21.15.31:0/2387786846 conn(0x7f5b140fc420 msgr2=0x7f5b14104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.841+0000 7f5b19587640 1 -- 172.21.15.31:0/2387786846 shutdown_connections 2024-04-25T13:17:44.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.842+0000 7f5b19587640 1 -- 172.21.15.31:0/2387786846 wait complete. 2024-04-25T13:17:44.856 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T13:17:44.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T13:17:44.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.966+0000 7f60bce59640 1 -- 172.21.15.31:0/2576750360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8102a30 msgr2=0x7f60b8104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:44.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.966+0000 7f60bce59640 1 --2- 172.21.15.31:0/2576750360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8102a30 0x7f60b8104e20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009900 tx=0x7f60a802a410 comp rx=0 tx=0).stop 2024-04-25T13:17:44.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.966+0000 7f60bce59640 1 -- 172.21.15.31:0/2576750360 shutdown_connections 2024-04-25T13:17:44.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.966+0000 7f60bce59640 1 --2- 172.21.15.31:0/2576750360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b8107c90 0x7f60b810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:44.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.966+0000 7f60bce59640 1 --2- 172.21.15.31:0/2576750360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60b8105360 0x7f60b8107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:44.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.966+0000 7f60bce59640 1 --2- 172.21.15.31:0/2576750360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8102a30 0x7f60b8104e20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:44.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.966+0000 7f60bce59640 1 -- 172.21.15.31:0/2576750360 >> 172.21.15.31:0/2576750360 conn(0x7f60b80fc420 msgr2=0x7f60b80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:44.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.966+0000 7f60bce59640 1 -- 172.21.15.31:0/2576750360 shutdown_connections 2024-04-25T13:17:44.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.966+0000 7f60bce59640 1 -- 172.21.15.31:0/2576750360 wait complete. 2024-04-25T13:17:44.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.967+0000 7f60bce59640 1 Processor -- start 2024-04-25T13:17:44.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.967+0000 7f60bce59640 1 -- start start 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.967+0000 7f60bce59640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60b8102a30 0x7f60b819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.967+0000 7f60bce59640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b8105360 0x7f60b819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.967+0000 7f60bce59640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107c90 0x7f60b81a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.967+0000 7f60bce59640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810db40 con 0x7f60b8107c90 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.967+0000 7f60bce59640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810d9c0 con 0x7f60b8105360 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.967+0000 7f60bce59640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810dcc0 con 0x7f60b8102a30 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.967+0000 7f60b6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107c90 0x7f60b81a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.967+0000 7f60b6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107c90 0x7f60b81a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50758/0 (socket says 172.21.15.31:50758) 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.967+0000 7f60b6d9d640 1 -- 172.21.15.31:0/1731954039 learned_addr learned my addr 172.21.15.31:0/1731954039 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60b659c640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60b8102a30 0x7f60b819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60b5d9b640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b8105360 0x7f60b819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:44.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60b6d9d640 1 -- 172.21.15.31:0/1731954039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60b8102a30 msgr2=0x7f60b819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:44.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60b6d9d640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60b8102a30 0x7f60b819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:44.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60b6d9d640 1 -- 172.21.15.31:0/1731954039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b8105360 msgr2=0x7f60b819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:44.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60b6d9d640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b8105360 0x7f60b819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:44.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60b6d9d640 1 -- 172.21.15.31:0/1731954039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b81a2080 con 0x7f60b8107c90 2024-04-25T13:17:44.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60b6d9d640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107c90 0x7f60b81a1a10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f60b000cad0 tx=0x7f60b000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:44.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60a77fe640 1 -- 172.21.15.31:0/1731954039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b000ae20 con 0x7f60b8107c90 2024-04-25T13:17:44.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60a77fe640 1 -- 172.21.15.31:0/1731954039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b0005020 con 0x7f60b8107c90 2024-04-25T13:17:44.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60bce59640 1 -- 172.21.15.31:0/1731954039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b81a2370 con 0x7f60b8107c90 2024-04-25T13:17:44.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60a77fe640 1 -- 172.21.15.31:0/1731954039 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b000f6b0 con 0x7f60b8107c90 2024-04-25T13:17:44.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.968+0000 7f60bce59640 1 -- 172.21.15.31:0/1731954039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b81a28c0 con 0x7f60b8107c90 2024-04-25T13:17:44.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.969+0000 7f60bce59640 1 -- 172.21.15.31:0/1731954039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b81097e0 con 0x7f60b8107c90 2024-04-25T13:17:44.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.970+0000 7f60a77fe640 1 -- 172.21.15.31:0/1731954039 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f60b00051d0 con 0x7f60b8107c90 2024-04-25T13:17:44.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.971+0000 7f60a77fe640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60a00a3c30 0x7f60a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:44.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.971+0000 7f60b659c640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60a00a3c30 0x7f60a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:44.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.971+0000 7f60a77fe640 1 -- 172.21.15.31:0/1731954039 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f60b00d02f0 con 0x7f60b8107c90 2024-04-25T13:17:44.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.971+0000 7f60b659c640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60a00a3c30 0x7f60a00a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f60a8031500 tx=0x7f60a8005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:44.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:44.975+0000 7f60a77fe640 1 -- 172.21.15.31:0/1731954039 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60b0014030 con 0x7f60b8107c90 2024-04-25T13:17:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:44 smithi066 ceph-mon[33137]: pgmap v294: 97 pgs: 11 creating+peering, 76 active+clean, 10 unknown; 578 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 3 op/s 2024-04-25T13:17:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:44 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/542541087' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:17:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:44 smithi066 ceph-mon[33137]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:44 smithi066 ceph-mon[33137]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:17:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:44 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2387786846' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:17:45.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.188+0000 7f60bce59640 1 -- 172.21.15.31:0/1731954039 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f60b8062390 con 0x7f60a00a3c30 2024-04-25T13:17:45.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:44 smithi031 ceph-mon[29457]: pgmap v294: 97 pgs: 11 creating+peering, 76 active+clean, 10 unknown; 578 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 3 op/s 2024-04-25T13:17:45.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:44 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/542541087' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:17:45.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:44 smithi031 ceph-mon[29457]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:45.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:44 smithi031 ceph-mon[29457]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:17:45.195 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:44 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2387786846' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:17:45.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.209+0000 7f60a77fe640 1 -- 172.21.15.31:0/1731954039 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f60b8062390 con 0x7f60a00a3c30 2024-04-25T13:17:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 -- 172.21.15.31:0/1731954039 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60a00a3c30 msgr2=0x7f60a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60a00a3c30 0x7f60a00a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f60a8031500 tx=0x7f60a8005a20 comp rx=0 tx=0).stop 2024-04-25T13:17:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 -- 172.21.15.31:0/1731954039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107c90 msgr2=0x7f60b81a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107c90 0x7f60b81a1a10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f60b000cad0 tx=0x7f60b000cf90 comp rx=0 tx=0).stop 2024-04-25T13:17:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 -- 172.21.15.31:0/1731954039 shutdown_connections 2024-04-25T13:17:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60a00a3c30 0x7f60a00a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107c90 0x7f60b81a1a10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b8105360 0x7f60b819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 --2- 172.21.15.31:0/1731954039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60b8102a30 0x7f60b819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 -- 172.21.15.31:0/1731954039 >> 172.21.15.31:0/1731954039 conn(0x7f60b80fc420 msgr2=0x7f60b81021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 -- 172.21.15.31:0/1731954039 shutdown_connections 2024-04-25T13:17:45.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.211+0000 7f60bce59640 1 -- 172.21.15.31:0/1731954039 wait complete. 2024-04-25T13:17:45.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:44 smithi136 ceph-mon[33038]: pgmap v294: 97 pgs: 11 creating+peering, 76 active+clean, 10 unknown; 578 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 3 op/s 2024-04-25T13:17:45.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:44 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/542541087' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:17:45.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:44 smithi136 ceph-mon[33038]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T13:17:45.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:44 smithi136 ceph-mon[33038]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:17:45.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:44 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2387786846' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:17:45.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T13:17:45.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.329+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/946807515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb0101fe0 msgr2=0x7f8eb0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.329+0000 7f8eb4ecf640 1 --2- 172.21.15.31:0/946807515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb0101fe0 0x7f8eb0102460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009900 tx=0x7f8ea402a410 comp rx=0 tx=0).stop 2024-04-25T13:17:45.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.329+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/946807515 shutdown_connections 2024-04-25T13:17:45.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.329+0000 7f8eb4ecf640 1 --2- 172.21.15.31:0/946807515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb01029a0 0x7f8eb01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.329+0000 7f8eb4ecf640 1 --2- 172.21.15.31:0/946807515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb0101fe0 0x7f8eb0102460 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.329+0000 7f8eb4ecf640 1 --2- 172.21.15.31:0/946807515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb0100e90 0x7f8eb01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.329+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/946807515 >> 172.21.15.31:0/946807515 conn(0x7f8eb00fc440 msgr2=0x7f8eb00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:45.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.329+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/946807515 shutdown_connections 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.329+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/946807515 wait complete. 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.329+0000 7f8eb4ecf640 1 Processor -- start 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eb4ecf640 1 -- start start 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eb4ecf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb0100e90 0x7f8eb01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eb4ecf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb0101fe0 0x7f8eb0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eb4ecf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb01029a0 0x7f8eb019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eb4ecf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb010ca40 con 0x7f8eb01029a0 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eb4ecf640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb010c8c0 con 0x7f8eb0101fe0 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eb4ecf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb010cbc0 con 0x7f8eb0100e90 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eaed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb01029a0 0x7f8eb019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eaed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb01029a0 0x7f8eb019f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50786/0 (socket says 172.21.15.31:50786) 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eaed9d640 1 -- 172.21.15.31:0/2446019310 learned_addr learned my addr 172.21.15.31:0/2446019310 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eadd9b640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb0101fe0 0x7f8eb0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eae59c640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb0100e90 0x7f8eb01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:45.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eaed9d640 1 -- 172.21.15.31:0/2446019310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb0100e90 msgr2=0x7f8eb01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eaed9d640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb0100e90 0x7f8eb01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eaed9d640 1 -- 172.21.15.31:0/2446019310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb0101fe0 msgr2=0x7f8eb0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eaed9d640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb0101fe0 0x7f8eb0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eaed9d640 1 -- 172.21.15.31:0/2446019310 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb019ff10 con 0x7f8eb01029a0 2024-04-25T13:17:45.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eaed9d640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb01029a0 0x7f8eb019f8a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800dbd0 tx=0x7f8ea800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:45.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8e9f7fe640 1 -- 172.21.15.31:0/2446019310 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8004b90 con 0x7f8eb01029a0 2024-04-25T13:17:45.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8e9f7fe640 1 -- 172.21.15.31:0/2446019310 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea8004d30 con 0x7f8eb01029a0 2024-04-25T13:17:45.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/2446019310 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb01a0200 con 0x7f8eb01029a0 2024-04-25T13:17:45.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.330+0000 7f8e9f7fe640 1 -- 172.21.15.31:0/2446019310 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8004720 con 0x7f8eb01029a0 2024-04-25T13:17:45.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.331+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/2446019310 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb01a0770 con 0x7f8eb01029a0 2024-04-25T13:17:45.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.331+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/2446019310 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb01a09d0 con 0x7f8eb01029a0 2024-04-25T13:17:45.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.333+0000 7f8e9f7fe640 1 -- 172.21.15.31:0/2446019310 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8ea80039c0 con 0x7f8eb01029a0 2024-04-25T13:17:45.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.333+0000 7f8e9f7fe640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8e980a3c30 0x7f8e980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:45.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.333+0000 7f8eae59c640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8e980a3c30 0x7f8e980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:45.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.333+0000 7f8e9f7fe640 1 -- 172.21.15.31:0/2446019310 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f8ea80d0ce0 con 0x7f8eb01029a0 2024-04-25T13:17:45.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.334+0000 7f8eae59c640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8e980a3c30 0x7f8e980a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009890 tx=0x7f8ea0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:45.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.337+0000 7f8e9f7fe640 1 -- 172.21.15.31:0/2446019310 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ea8017030 con 0x7f8eb01029a0 2024-04-25T13:17:45.483 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:17:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:17:45.193+0000 7f7976624700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:17:45.484 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:17:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:17:45.193+0000 7f7976624700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:17:45.484 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:17:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:17:45.193+0000 7f7976624700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:17:45.484 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:17:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:17:45.193+0000 7f7976624700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:17:45.484 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:17:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:17:45.193+0000 7f7976624700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:17:45.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.551+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/2446019310 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8eb006ad50 con 0x7f8e980a3c30 2024-04-25T13:17:45.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.577+0000 7f8e9f7fe640 1 -- 172.21.15.31:0/2446019310 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8eb006ad50 con 0x7f8e980a3c30 2024-04-25T13:17:45.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/2446019310 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8e980a3c30 msgr2=0x7f8e980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8e980a3c30 0x7f8e980a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009890 tx=0x7f8ea0008040 comp rx=0 tx=0).stop 2024-04-25T13:17:45.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/2446019310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb01029a0 msgr2=0x7f8eb019f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb01029a0 0x7f8eb019f8a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800dbd0 tx=0x7f8ea800b260 comp rx=0 tx=0).stop 2024-04-25T13:17:45.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/2446019310 shutdown_connections 2024-04-25T13:17:45.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f8e980a3c30 0x7f8e980a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb01029a0 0x7f8eb019f8a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8eb0101fe0 0x7f8eb0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 --2- 172.21.15.31:0/2446019310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb0100e90 0x7f8eb01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/2446019310 >> 172.21.15.31:0/2446019310 conn(0x7f8eb00fc440 msgr2=0x7f8eb006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/2446019310 shutdown_connections 2024-04-25T13:17:45.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.579+0000 7f8eb4ecf640 1 -- 172.21.15.31:0/2446019310 wait complete. 2024-04-25T13:17:45.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T13:17:45.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.697+0000 7f52e2dbf640 1 -- 172.21.15.31:0/1516315842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc102a70 msgr2=0x7f52dc104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.697+0000 7f52e2dbf640 1 --2- 172.21.15.31:0/1516315842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc102a70 0x7f52dc104e60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009900 tx=0x7f52d002a410 comp rx=0 tx=0).stop 2024-04-25T13:17:45.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.697+0000 7f52e2dbf640 1 -- 172.21.15.31:0/1516315842 shutdown_connections 2024-04-25T13:17:45.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.697+0000 7f52e2dbf640 1 --2- 172.21.15.31:0/1516315842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52dc107cd0 0x7f52dc10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.697+0000 7f52e2dbf640 1 --2- 172.21.15.31:0/1516315842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52dc1053a0 0x7f52dc107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.697+0000 7f52e2dbf640 1 --2- 172.21.15.31:0/1516315842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc102a70 0x7f52dc104e60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.697+0000 7f52e2dbf640 1 -- 172.21.15.31:0/1516315842 >> 172.21.15.31:0/1516315842 conn(0x7f52dc0fc400 msgr2=0x7f52dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.697+0000 7f52e2dbf640 1 -- 172.21.15.31:0/1516315842 shutdown_connections 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.697+0000 7f52e2dbf640 1 -- 172.21.15.31:0/1516315842 wait complete. 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52e2dbf640 1 Processor -- start 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52e2dbf640 1 -- start start 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52e2dbf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52dc102a70 0x7f52dc1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52e2dbf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1053a0 0x7f52dc1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52e2dbf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52dc107cd0 0x7f52dc19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52e2dbf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc10db10 con 0x7f52dc1053a0 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52e2dbf640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc10d990 con 0x7f52dc107cd0 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52e2dbf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc10dc90 con 0x7f52dc102a70 2024-04-25T13:17:45.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1053a0 0x7f52dc1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1053a0 0x7f52dc1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50804/0 (socket says 172.21.15.31:50804) 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52dbfff640 1 -- 172.21.15.31:0/3314415951 learned_addr learned my addr 172.21.15.31:0/3314415951 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52e135c640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52dc107cd0 0x7f52dc19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52e0b5b640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52dc102a70 0x7f52dc1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52dbfff640 1 -- 172.21.15.31:0/3314415951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52dc102a70 msgr2=0x7f52dc1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52dbfff640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52dc102a70 0x7f52dc1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52dbfff640 1 -- 172.21.15.31:0/3314415951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52dc107cd0 msgr2=0x7f52dc19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52dbfff640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52dc107cd0 0x7f52dc19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.698+0000 7f52dbfff640 1 -- 172.21.15.31:0/3314415951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc19fe80 con 0x7f52dc1053a0 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.699+0000 7f52dbfff640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1053a0 0x7f52dc1987f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f52c800e9c0 tx=0x7f52c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.699+0000 7f52d9ffb640 1 -- 172.21.15.31:0/3314415951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c800cd80 con 0x7f52dc1053a0 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.699+0000 7f52d9ffb640 1 -- 172.21.15.31:0/3314415951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52c80054d0 con 0x7f52dc1053a0 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.699+0000 7f52e2dbf640 1 -- 172.21.15.31:0/3314415951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc1a0170 con 0x7f52dc1053a0 2024-04-25T13:17:45.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.699+0000 7f52d9ffb640 1 -- 172.21.15.31:0/3314415951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8010430 con 0x7f52dc1053a0 2024-04-25T13:17:45.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.699+0000 7f52e2dbf640 1 -- 172.21.15.31:0/3314415951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc1a0710 con 0x7f52dc1053a0 2024-04-25T13:17:45.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.700+0000 7f52e2dbf640 1 -- 172.21.15.31:0/3314415951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52dc0fdef0 con 0x7f52dc1053a0 2024-04-25T13:17:45.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.701+0000 7f52d9ffb640 1 -- 172.21.15.31:0/3314415951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f52c8004fd0 con 0x7f52dc1053a0 2024-04-25T13:17:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.701+0000 7f52d9ffb640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f52cc0a3c30 0x7f52cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.702+0000 7f52e0b5b640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f52cc0a3c30 0x7f52cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.702+0000 7f52d9ffb640 1 -- 172.21.15.31:0/3314415951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f52c80d0740 con 0x7f52dc1053a0 2024-04-25T13:17:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.702+0000 7f52e0b5b640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f52cc0a3c30 0x7f52cc0a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f52d0031500 tx=0x7f52d0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:45.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.706+0000 7f52d9ffb640 1 -- 172.21.15.31:0/3314415951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52c809d1d0 con 0x7f52dc1053a0 2024-04-25T13:17:45.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.921+0000 7f52e2dbf640 1 -- 172.21.15.31:0/3314415951 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f52dc062390 con 0x7f52cc0a3c30 2024-04-25T13:17:45.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.927+0000 7f52d9ffb640 1 -- 172.21.15.31:0/3314415951 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f52dc062390 con 0x7f52cc0a3c30 2024-04-25T13:17:45.928 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_0/4613773b-ec1e-44a5-8e66-922fe92629a8 2024-04-25T13:17:45.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 -- 172.21.15.31:0/3314415951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f52cc0a3c30 msgr2=0x7f52cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f52cc0a3c30 0x7f52cc0a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f52d0031500 tx=0x7f52d0005fd0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 -- 172.21.15.31:0/3314415951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1053a0 msgr2=0x7f52dc1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1053a0 0x7f52dc1987f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f52c800e9c0 tx=0x7f52c800ee80 comp rx=0 tx=0).stop 2024-04-25T13:17:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 -- 172.21.15.31:0/3314415951 shutdown_connections 2024-04-25T13:17:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f52cc0a3c30 0x7f52cc0a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52dc107cd0 0x7f52dc19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52dc1053a0 0x7f52dc1987f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 --2- 172.21.15.31:0/3314415951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52dc102a70 0x7f52dc1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 -- 172.21.15.31:0/3314415951 >> 172.21.15.31:0/3314415951 conn(0x7f52dc0fc400 msgr2=0x7f52dc103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 -- 172.21.15.31:0/3314415951 shutdown_connections 2024-04-25T13:17:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:45.929+0000 7f52e2dbf640 1 -- 172.21.15.31:0/3314415951 wait complete. 2024-04-25T13:17:45.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T13:17:46.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.047+0000 7fb506987640 1 -- 172.21.15.31:0/2289443605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb500100e90 msgr2=0x7fb5001012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.047+0000 7fb506987640 1 --2- 172.21.15.31:0/2289443605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb500100e90 0x7fb5001012b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009900 tx=0x7fb4f002a410 comp rx=0 tx=0).stop 2024-04-25T13:17:46.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.047+0000 7fb506987640 1 -- 172.21.15.31:0/2289443605 shutdown_connections 2024-04-25T13:17:46.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.047+0000 7fb506987640 1 --2- 172.21.15.31:0/2289443605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5001029a0 0x7fb5001092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.047+0000 7fb506987640 1 --2- 172.21.15.31:0/2289443605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb500101fe0 0x7fb500102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.047+0000 7fb506987640 1 --2- 172.21.15.31:0/2289443605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb500100e90 0x7fb5001012b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.047+0000 7fb506987640 1 -- 172.21.15.31:0/2289443605 >> 172.21.15.31:0/2289443605 conn(0x7fb5000fc400 msgr2=0x7fb5000fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:46.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.047+0000 7fb506987640 1 -- 172.21.15.31:0/2289443605 shutdown_connections 2024-04-25T13:17:46.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.047+0000 7fb506987640 1 -- 172.21.15.31:0/2289443605 wait complete. 2024-04-25T13:17:46.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.047+0000 7fb506987640 1 Processor -- start 2024-04-25T13:17:46.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.047+0000 7fb506987640 1 -- start start 2024-04-25T13:17:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb506987640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb500100e90 0x7fb50010b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb506987640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb500101fe0 0x7fb500109e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb506987640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5001029a0 0x7fb50010a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb506987640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb50010db30 con 0x7fb500100e90 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb506987640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb50010d9b0 con 0x7fb500101fe0 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb506987640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb50010dcb0 con 0x7fb5001029a0 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb500100e90 0x7fb50010b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb500100e90 0x7fb50010b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50824/0 (socket says 172.21.15.31:50824) 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fffff640 1 -- 172.21.15.31:0/579286840 learned_addr learned my addr 172.21.15.31:0/579286840 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4ff7fe640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb500101fe0 0x7fb500109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb504f24640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5001029a0 0x7fb50010a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fffff640 1 -- 172.21.15.31:0/579286840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5001029a0 msgr2=0x7fb50010a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fffff640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5001029a0 0x7fb50010a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fffff640 1 -- 172.21.15.31:0/579286840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb500101fe0 msgr2=0x7fb500109e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fffff640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb500101fe0 0x7fb500109e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fffff640 1 -- 172.21.15.31:0/579286840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50010ac40 con 0x7fb500100e90 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb504f24640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5001029a0 0x7fb50010a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:46.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fffff640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb500100e90 0x7fb50010b830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0005940 tx=0x7fb4f0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:46.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fd7fa640 1 -- 172.21.15.31:0/579286840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f003f030 con 0x7fb500100e90 2024-04-25T13:17:46.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fd7fa640 1 -- 172.21.15.31:0/579286840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f0003aa0 con 0x7fb500100e90 2024-04-25T13:17:46.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb506987640 1 -- 172.21.15.31:0/579286840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5001a22f0 con 0x7fb500100e90 2024-04-25T13:17:46.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb4fd7fa640 1 -- 172.21.15.31:0/579286840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0035070 con 0x7fb500100e90 2024-04-25T13:17:46.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.048+0000 7fb506987640 1 -- 172.21.15.31:0/579286840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5001a2840 con 0x7fb500100e90 2024-04-25T13:17:46.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.049+0000 7fb506987640 1 -- 172.21.15.31:0/579286840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5001082c0 con 0x7fb500100e90 2024-04-25T13:17:46.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.050+0000 7fb4fd7fa640 1 -- 172.21.15.31:0/579286840 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb4f0042030 con 0x7fb500100e90 2024-04-25T13:17:46.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.051+0000 7fb4fd7fa640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3bb0 0x7fb4e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.051+0000 7fb4ff7fe640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3bb0 0x7fb4e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.051+0000 7fb4fd7fa640 1 -- 172.21.15.31:0/579286840 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb4f00f0990 con 0x7fb500100e90 2024-04-25T13:17:46.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.051+0000 7fb4ff7fe640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3bb0 0x7fb4e80a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb50010b5a0 tx=0x7fb4f400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:46.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.055+0000 7fb4fd7fa640 1 -- 172.21.15.31:0/579286840 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4f00bd420 con 0x7fb500100e90 2024-04-25T13:17:46.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:45 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2387786846' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:17:46.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:45 smithi066 ceph-mon[33137]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:17:46.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:45 smithi066 ceph-mon[33137]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:46.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:45 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2387786846' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:17:46.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:45 smithi136 ceph-mon[33038]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:17:46.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:45 smithi136 ceph-mon[33038]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:46.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:45 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2387786846' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:17:46.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:45 smithi031 ceph-mon[29457]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:17:46.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:45 smithi031 ceph-mon[29457]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:46.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.273+0000 7fb506987640 1 -- 172.21.15.31:0/579286840 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb500062420 con 0x7fb4e80a3bb0 2024-04-25T13:17:46.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.299+0000 7fb4fd7fa640 1 -- 172.21.15.31:0/579286840 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb500062420 con 0x7fb4e80a3bb0 2024-04-25T13:17:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 -- 172.21.15.31:0/579286840 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3bb0 msgr2=0x7fb4e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3bb0 0x7fb4e80a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb50010b5a0 tx=0x7fb4f400b390 comp rx=0 tx=0).stop 2024-04-25T13:17:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 -- 172.21.15.31:0/579286840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb500100e90 msgr2=0x7fb50010b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb500100e90 0x7fb50010b830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0005940 tx=0x7fb4f0031a50 comp rx=0 tx=0).stop 2024-04-25T13:17:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 -- 172.21.15.31:0/579286840 shutdown_connections 2024-04-25T13:17:46.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3bb0 0x7fb4e80a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5001029a0 0x7fb50010a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb500101fe0 0x7fb500109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 --2- 172.21.15.31:0/579286840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb500100e90 0x7fb50010b830 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 -- 172.21.15.31:0/579286840 >> 172.21.15.31:0/579286840 conn(0x7fb5000fc400 msgr2=0x7fb500105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 -- 172.21.15.31:0/579286840 shutdown_connections 2024-04-25T13:17:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.301+0000 7fb506987640 1 -- 172.21.15.31:0/579286840 wait complete. 2024-04-25T13:17:46.313 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T13:17:46.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.419+0000 7f35d7a72640 1 -- 172.21.15.31:0/2290497545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0100e60 msgr2=0x7f35d0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.419+0000 7f35d7a72640 1 --2- 172.21.15.31:0/2290497545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0100e60 0x7f35d0101280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009900 tx=0x7f35cc02a410 comp rx=0 tx=0).stop 2024-04-25T13:17:46.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.419+0000 7f35d7a72640 1 -- 172.21.15.31:0/2290497545 shutdown_connections 2024-04-25T13:17:46.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.419+0000 7f35d7a72640 1 --2- 172.21.15.31:0/2290497545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d0102a00 0x7f35d0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.419+0000 7f35d7a72640 1 --2- 172.21.15.31:0/2290497545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d0102040 0x7f35d01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.419+0000 7f35d7a72640 1 --2- 172.21.15.31:0/2290497545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0100e60 0x7f35d0101280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.419+0000 7f35d7a72640 1 -- 172.21.15.31:0/2290497545 >> 172.21.15.31:0/2290497545 conn(0x7f35d00fc440 msgr2=0x7f35d00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:46.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.419+0000 7f35d7a72640 1 -- 172.21.15.31:0/2290497545 shutdown_connections 2024-04-25T13:17:46.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.419+0000 7f35d7a72640 1 -- 172.21.15.31:0/2290497545 wait complete. 2024-04-25T13:17:46.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.419+0000 7f35d7a72640 1 Processor -- start 2024-04-25T13:17:46.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.419+0000 7f35d7a72640 1 -- start start 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d7a72640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d0100e60 0x7f35d01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d7a72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0102040 0x7f35d01987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d7a72640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d0102a00 0x7f35d019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d7a72640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f35d010db70 con 0x7f35d0102040 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d7a72640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f35d010d9f0 con 0x7f35d0102a00 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d7a72640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f35d010dcf0 con 0x7f35d0100e60 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d500d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0102040 0x7f35d01987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d500d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0102040 0x7f35d01987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50844/0 (socket says 172.21.15.31:50844) 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d500d640 1 -- 172.21.15.31:0/306183463 learned_addr learned my addr 172.21.15.31:0/306183463 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d600f640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d0102a00 0x7f35d019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d580e640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d0100e60 0x7f35d01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d500d640 1 -- 172.21.15.31:0/306183463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d0100e60 msgr2=0x7f35d01982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d500d640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d0100e60 0x7f35d01982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d500d640 1 -- 172.21.15.31:0/306183463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d0102a00 msgr2=0x7f35d019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d500d640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d0102a00 0x7f35d019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d500d640 1 -- 172.21.15.31:0/306183463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d019fec0 con 0x7f35d0102040 2024-04-25T13:17:46.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d500d640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0102040 0x7f35d01987e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f35c000d950 tx=0x7f35c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:46.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35c6ffd640 1 -- 172.21.15.31:0/306183463 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0017030 con 0x7f35d0102040 2024-04-25T13:17:46.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35c6ffd640 1 -- 172.21.15.31:0/306183463 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c0004020 con 0x7f35d0102040 2024-04-25T13:17:46.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.420+0000 7f35d7a72640 1 -- 172.21.15.31:0/306183463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d01a01b0 con 0x7f35d0102040 2024-04-25T13:17:46.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.421+0000 7f35c6ffd640 1 -- 172.21.15.31:0/306183463 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c00047a0 con 0x7f35d0102040 2024-04-25T13:17:46.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.421+0000 7f35d7a72640 1 -- 172.21.15.31:0/306183463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d01a0750 con 0x7f35d0102040 2024-04-25T13:17:46.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.421+0000 7f35d7a72640 1 -- 172.21.15.31:0/306183463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d0108270 con 0x7f35d0102040 2024-04-25T13:17:46.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.423+0000 7f35c6ffd640 1 -- 172.21.15.31:0/306183463 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f35c0009a30 con 0x7f35d0102040 2024-04-25T13:17:46.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.423+0000 7f35c6ffd640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f35bc0a3c30 0x7f35bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.423+0000 7f35c6ffd640 1 -- 172.21.15.31:0/306183463 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f35c00cfd10 con 0x7f35d0102040 2024-04-25T13:17:46.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.423+0000 7f35d580e640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f35bc0a3c30 0x7f35bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.424+0000 7f35d580e640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f35bc0a3c30 0x7f35bc0a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f35cc000c00 tx=0x7f35cc0314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:46.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.428+0000 7f35c6ffd640 1 -- 172.21.15.31:0/306183463 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35c009c840 con 0x7f35d0102040 2024-04-25T13:17:46.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.644+0000 7f35d7a72640 1 -- 172.21.15.31:0/306183463 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f35d0062390 con 0x7f35bc0a3c30 2024-04-25T13:17:46.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.650+0000 7f35c6ffd640 1 -- 172.21.15.31:0/306183463 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f35d0062390 con 0x7f35bc0a3c30 2024-04-25T13:17:46.650 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_1/4313c4b3-3c68-4ee3-ad7e-c53ba53f1895 2024-04-25T13:17:46.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.651+0000 7f35d7a72640 1 -- 172.21.15.31:0/306183463 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f35bc0a3c30 msgr2=0x7f35bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.651+0000 7f35d7a72640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f35bc0a3c30 0x7f35bc0a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f35cc000c00 tx=0x7f35cc0314a0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.651+0000 7f35d7a72640 1 -- 172.21.15.31:0/306183463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0102040 msgr2=0x7f35d01987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.651+0000 7f35d7a72640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0102040 0x7f35d01987e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f35c000d950 tx=0x7f35c000de10 comp rx=0 tx=0).stop 2024-04-25T13:17:46.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.652+0000 7f35d7a72640 1 -- 172.21.15.31:0/306183463 shutdown_connections 2024-04-25T13:17:46.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.652+0000 7f35d7a72640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f35bc0a3c30 0x7f35bc0a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.652+0000 7f35d7a72640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f35d0102a00 0x7f35d019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.652+0000 7f35d7a72640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35d0102040 0x7f35d01987e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.652+0000 7f35d7a72640 1 --2- 172.21.15.31:0/306183463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d0100e60 0x7f35d01982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.652+0000 7f35d7a72640 1 -- 172.21.15.31:0/306183463 >> 172.21.15.31:0/306183463 conn(0x7f35d00fc440 msgr2=0x7f35d0105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:46.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.652+0000 7f35d7a72640 1 -- 172.21.15.31:0/306183463 shutdown_connections 2024-04-25T13:17:46.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.652+0000 7f35d7a72640 1 -- 172.21.15.31:0/306183463 wait complete. 2024-04-25T13:17:46.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:17:46.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.764+0000 7f1ce973d640 1 -- 172.21.15.31:0/2435379800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce4102a80 msgr2=0x7f1ce4104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.764+0000 7f1ce973d640 1 --2- 172.21.15.31:0/2435379800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce4102a80 0x7f1ce4104e70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009900 tx=0x7f1cd402a410 comp rx=0 tx=0).stop 2024-04-25T13:17:46.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.764+0000 7f1ce973d640 1 -- 172.21.15.31:0/2435379800 shutdown_connections 2024-04-25T13:17:46.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.764+0000 7f1ce973d640 1 --2- 172.21.15.31:0/2435379800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ce4107ce0 0x7f1ce410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.764+0000 7f1ce973d640 1 --2- 172.21.15.31:0/2435379800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce41053b0 0x7f1ce41077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.764+0000 7f1ce973d640 1 --2- 172.21.15.31:0/2435379800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce4102a80 0x7f1ce4104e70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.764+0000 7f1ce973d640 1 -- 172.21.15.31:0/2435379800 >> 172.21.15.31:0/2435379800 conn(0x7f1ce40fc450 msgr2=0x7f1ce40fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:46.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.764+0000 7f1ce973d640 1 -- 172.21.15.31:0/2435379800 shutdown_connections 2024-04-25T13:17:46.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.764+0000 7f1ce973d640 1 -- 172.21.15.31:0/2435379800 wait complete. 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce973d640 1 Processor -- start 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce973d640 1 -- start start 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce973d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce4102a80 0x7f1ce4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce973d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce41053b0 0x7f1ce41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce973d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ce4107ce0 0x7f1ce419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce973d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce410dae0 con 0x7f1ce41053b0 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce973d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce410d960 con 0x7f1ce4107ce0 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce973d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce410dc60 con 0x7f1ce4102a80 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce41053b0 0x7f1ce41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce41053b0 0x7f1ce41986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50866/0 (socket says 172.21.15.31:50866) 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce27fc640 1 -- 172.21.15.31:0/910220169 learned_addr learned my addr 172.21.15.31:0/910220169 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce2ffd640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce4102a80 0x7f1ce4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce37fe640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ce4107ce0 0x7f1ce419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce27fc640 1 -- 172.21.15.31:0/910220169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce4102a80 msgr2=0x7f1ce4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce27fc640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce4102a80 0x7f1ce4198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce27fc640 1 -- 172.21.15.31:0/910220169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ce4107ce0 msgr2=0x7f1ce419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:46.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce27fc640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ce4107ce0 0x7f1ce419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:46.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce27fc640 1 -- 172.21.15.31:0/910220169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce419fd90 con 0x7f1ce41053b0 2024-04-25T13:17:46.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce37fe640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ce4107ce0 0x7f1ce419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:46.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.765+0000 7f1ce27fc640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce41053b0 0x7f1ce41986b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8005fd0 tx=0x7f1cd800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:46.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.766+0000 7f1ccbfff640 1 -- 172.21.15.31:0/910220169 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8017070 con 0x7f1ce41053b0 2024-04-25T13:17:46.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.766+0000 7f1ce973d640 1 -- 172.21.15.31:0/910220169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce41a0080 con 0x7f1ce41053b0 2024-04-25T13:17:46.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.766+0000 7f1ccbfff640 1 -- 172.21.15.31:0/910220169 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd8009c30 con 0x7f1ce41053b0 2024-04-25T13:17:46.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.766+0000 7f1ccbfff640 1 -- 172.21.15.31:0/910220169 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8003d30 con 0x7f1ce41053b0 2024-04-25T13:17:46.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.766+0000 7f1ce973d640 1 -- 172.21.15.31:0/910220169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce41a05d0 con 0x7f1ce41053b0 2024-04-25T13:17:46.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.767+0000 7f1ce973d640 1 -- 172.21.15.31:0/910220169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce40fdf40 con 0x7f1ce41053b0 2024-04-25T13:17:46.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.768+0000 7f1ccbfff640 1 -- 172.21.15.31:0/910220169 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1cd8003ed0 con 0x7f1ce41053b0 2024-04-25T13:17:46.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.768+0000 7f1ccbfff640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1ccc0a3bb0 0x7f1ccc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:46.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.768+0000 7f1ce2ffd640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1ccc0a3bb0 0x7f1ccc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:46.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.768+0000 7f1ccbfff640 1 -- 172.21.15.31:0/910220169 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f1cd80d0060 con 0x7f1ce41053b0 2024-04-25T13:17:46.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.769+0000 7f1ce2ffd640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1ccc0a3bb0 0x7f1ccc0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4005c40 tx=0x7f1cd4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.773+0000 7f1ccbfff640 1 -- 172.21.15.31:0/910220169 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1cd8020030 con 0x7f1ce41053b0 2024-04-25T13:17:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:46.851+0000 7f1ccbfff640 1 -- 172.21.15.31:0/910220169 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1cd8098b00 con 0x7f1ce41053b0 2024-04-25T13:17:46.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:46 smithi031 ceph-mon[29457]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:46.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:46 smithi031 ceph-mon[29457]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T13:17:46.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:46 smithi031 ceph-mon[29457]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:46.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:46 smithi031 ceph-mon[29457]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:47.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.003+0000 7f1ce973d640 1 -- 172.21.15.31:0/910220169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ce402d830 con 0x7f1ce41053b0 2024-04-25T13:17:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.003+0000 7f1ccbfff640 1 -- 172.21.15.31:0/910220169 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11798 (secure 0 0 0) 0x7f1cd809cb90 con 0x7f1ce41053b0 2024-04-25T13:17:47.004 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:47.005 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2599371925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2599371925},{"type":"v1","addr":"172.21.15.31:6839","nonce":2599371925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/164386980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":164386980},{"type":"v1","addr":"172.21.15.31:6841","nonce":164386980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:17:40.162672+0000","modified":"2024-04-25T13:17:44.194940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2682191586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2682191586},{"type":"v1","addr":"172.21.15.66:6839","nonce":2682191586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24511]},"id":1}]} 2024-04-25T13:17:47.005 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-04-25T13:17:47.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.005+0000 7f1ce973d640 1 -- 172.21.15.31:0/910220169 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1ccc0a3bb0 msgr2=0x7f1ccc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.005+0000 7f1ce973d640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1ccc0a3bb0 0x7f1ccc0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4005c40 tx=0x7f1cd4000bc0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.005+0000 7f1ce973d640 1 -- 172.21.15.31:0/910220169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce41053b0 msgr2=0x7f1ce41986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.005+0000 7f1ce973d640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce41053b0 0x7f1ce41986b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8005fd0 tx=0x7f1cd800dde0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.005+0000 7f1ce973d640 1 -- 172.21.15.31:0/910220169 shutdown_connections 2024-04-25T13:17:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.005+0000 7f1ce973d640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1ccc0a3bb0 0x7f1ccc0a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.005+0000 7f1ce973d640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ce4107ce0 0x7f1ce419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.005+0000 7f1ce973d640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce41053b0 0x7f1ce41986b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.005+0000 7f1ce973d640 1 --2- 172.21.15.31:0/910220169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ce4102a80 0x7f1ce4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.005+0000 7f1ce973d640 1 -- 172.21.15.31:0/910220169 >> 172.21.15.31:0/910220169 conn(0x7f1ce40fc450 msgr2=0x7f1ce41033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.006+0000 7f1ce973d640 1 -- 172.21.15.31:0/910220169 shutdown_connections 2024-04-25T13:17:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.006+0000 7f1ce973d640 1 -- 172.21.15.31:0/910220169 wait complete. 2024-04-25T13:17:47.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:17:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.118+0000 7f1e2c576640 1 -- 172.21.15.31:0/416604642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e24105340 msgr2=0x7f1e24107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.118+0000 7f1e2c576640 1 --2- 172.21.15.31:0/416604642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e24105340 0x7f1e24107730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009900 tx=0x7f1e1802a410 comp rx=0 tx=0).stop 2024-04-25T13:17:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.118+0000 7f1e2c576640 1 -- 172.21.15.31:0/416604642 shutdown_connections 2024-04-25T13:17:47.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.118+0000 7f1e2c576640 1 --2- 172.21.15.31:0/416604642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e24107c70 0x7f1e2410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.118+0000 7f1e2c576640 1 --2- 172.21.15.31:0/416604642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e24105340 0x7f1e24107730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.118+0000 7f1e2c576640 1 --2- 172.21.15.31:0/416604642 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e24102a10 0x7f1e24104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.118+0000 7f1e2c576640 1 -- 172.21.15.31:0/416604642 >> 172.21.15.31:0/416604642 conn(0x7f1e240fc400 msgr2=0x7f1e240fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.118+0000 7f1e2c576640 1 -- 172.21.15.31:0/416604642 shutdown_connections 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.118+0000 7f1e2c576640 1 -- 172.21.15.31:0/416604642 wait complete. 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e2c576640 1 Processor -- start 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e2c576640 1 -- start start 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e2c576640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e24102a10 0x7f1e241981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e2c576640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e24105340 0x7f1e24198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e2c576640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e24107c70 0x7f1e2419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e2c576640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2410db00 con 0x7f1e24105340 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e2c576640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2410d980 con 0x7f1e24107c70 2024-04-25T13:17:47.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e2c576640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2410dc80 con 0x7f1e24102a10 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e29b11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e24105340 0x7f1e24198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e29b11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e24105340 0x7f1e24198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50880/0 (socket says 172.21.15.31:50880) 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e29b11640 1 -- 172.21.15.31:0/2127621898 learned_addr learned my addr 172.21.15.31:0/2127621898 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e2a312640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e24102a10 0x7f1e241981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.119+0000 7f1e2ab13640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e24107c70 0x7f1e2419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.120+0000 7f1e29b11640 1 -- 172.21.15.31:0/2127621898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e24102a10 msgr2=0x7f1e241981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.120+0000 7f1e29b11640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e24102a10 0x7f1e241981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.120+0000 7f1e29b11640 1 -- 172.21.15.31:0/2127621898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e24107c70 msgr2=0x7f1e2419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.120+0000 7f1e29b11640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e24107c70 0x7f1e2419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.120+0000 7f1e29b11640 1 -- 172.21.15.31:0/2127621898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2419fdf0 con 0x7f1e24105340 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.120+0000 7f1e29b11640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e24105340 0x7f1e24198710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e18031500 tx=0x7f1e18003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.120+0000 7f1e237fe640 1 -- 172.21.15.31:0/2127621898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1803f070 con 0x7f1e24105340 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.120+0000 7f1e2c576640 1 -- 172.21.15.31:0/2127621898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e241a0080 con 0x7f1e24105340 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.120+0000 7f1e237fe640 1 -- 172.21.15.31:0/2127621898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e18003e70 con 0x7f1e24105340 2024-04-25T13:17:47.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.120+0000 7f1e237fe640 1 -- 172.21.15.31:0/2127621898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18035030 con 0x7f1e24105340 2024-04-25T13:17:47.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.120+0000 7f1e2c576640 1 -- 172.21.15.31:0/2127621898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e241a0570 con 0x7f1e24105340 2024-04-25T13:17:47.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.121+0000 7f1e2c576640 1 -- 172.21.15.31:0/2127621898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e240fdef0 con 0x7f1e24105340 2024-04-25T13:17:47.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.122+0000 7f1e237fe640 1 -- 172.21.15.31:0/2127621898 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1e18042070 con 0x7f1e24105340 2024-04-25T13:17:47.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.123+0000 7f1e237fe640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1e0c0a3c40 0x7f1e0c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.123+0000 7f1e2a312640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1e0c0a3c40 0x7f1e0c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.123+0000 7f1e237fe640 1 -- 172.21.15.31:0/2127621898 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f1e180f0fb0 con 0x7f1e24105340 2024-04-25T13:17:47.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.123+0000 7f1e2a312640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1e0c0a3c40 0x7f1e0c0a6100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00bbe0 tx=0x7f1e1c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:47.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.127+0000 7f1e237fe640 1 -- 172.21.15.31:0/2127621898 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e180bd390 con 0x7f1e24105340 2024-04-25T13:17:47.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:46 smithi066 ceph-mon[33137]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:47.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:46 smithi066 ceph-mon[33137]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T13:17:47.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:46 smithi066 ceph-mon[33137]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:47.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:46 smithi066 ceph-mon[33137]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:46 smithi136 ceph-mon[33038]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:46 smithi136 ceph-mon[33038]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T13:17:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:46 smithi136 ceph-mon[33038]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:46 smithi136 ceph-mon[33038]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:47.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.314+0000 7f1e2c576640 1 -- 172.21.15.31:0/2127621898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e2402d810 con 0x7f1e24105340 2024-04-25T13:17:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.316+0000 7f1e237fe640 1 -- 172.21.15.31:0/2127621898 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22182 (secure 0 0 0) 0x7f1e180c1690 con 0x7f1e24105340 2024-04-25T13:17:47.317 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:47.317 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:17:41.809574+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:17:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 -- 172.21.15.31:0/2127621898 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1e0c0a3c40 msgr2=0x7f1e0c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1e0c0a3c40 0x7f1e0c0a6100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00bbe0 tx=0x7f1e1c00b390 comp rx=0 tx=0).stop 2024-04-25T13:17:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 -- 172.21.15.31:0/2127621898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e24105340 msgr2=0x7f1e24198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e24105340 0x7f1e24198710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e18031500 tx=0x7f1e18003c80 comp rx=0 tx=0).stop 2024-04-25T13:17:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 -- 172.21.15.31:0/2127621898 shutdown_connections 2024-04-25T13:17:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1e0c0a3c40 0x7f1e0c0a6100 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e24107c70 0x7f1e2419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e24105340 0x7f1e24198710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 --2- 172.21.15.31:0/2127621898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e24102a10 0x7f1e241981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 -- 172.21.15.31:0/2127621898 >> 172.21.15.31:0/2127621898 conn(0x7f1e240fc400 msgr2=0x7f1e24103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 -- 172.21.15.31:0/2127621898 shutdown_connections 2024-04-25T13:17:47.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.318+0000 7f1e2c576640 1 -- 172.21.15.31:0/2127621898 wait complete. 2024-04-25T13:17:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:17:47.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:17:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.431+0000 7f18e898e640 1 -- 172.21.15.31:0/1422393533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e4100e90 msgr2=0x7f18e41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.431+0000 7f18e898e640 1 --2- 172.21.15.31:0/1422393533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e4100e90 0x7f18e41012b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009900 tx=0x7f18dc02a410 comp rx=0 tx=0).stop 2024-04-25T13:17:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.432+0000 7f18e898e640 1 -- 172.21.15.31:0/1422393533 shutdown_connections 2024-04-25T13:17:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.432+0000 7f18e898e640 1 --2- 172.21.15.31:0/1422393533 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e41029a0 0x7f18e41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.432+0000 7f18e898e640 1 --2- 172.21.15.31:0/1422393533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e4101fe0 0x7f18e4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.432+0000 7f18e898e640 1 --2- 172.21.15.31:0/1422393533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e4100e90 0x7f18e41012b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.432+0000 7f18e898e640 1 -- 172.21.15.31:0/1422393533 >> 172.21.15.31:0/1422393533 conn(0x7f18e40fc400 msgr2=0x7f18e40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.432+0000 7f18e898e640 1 -- 172.21.15.31:0/1422393533 shutdown_connections 2024-04-25T13:17:47.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.432+0000 7f18e898e640 1 -- 172.21.15.31:0/1422393533 wait complete. 2024-04-25T13:17:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.432+0000 7f18e898e640 1 Processor -- start 2024-04-25T13:17:47.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.432+0000 7f18e898e640 1 -- start start 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e898e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e4100e90 0x7f18e4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e898e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e4101fe0 0x7f18e41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e898e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e41029a0 0x7f18e419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e898e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f18e410db70 con 0x7f18e4101fe0 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e898e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18e410d9f0 con 0x7f18e41029a0 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e898e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18e410dcf0 con 0x7f18e4100e90 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e4101fe0 0x7f18e41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e4101fe0 0x7f18e41987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50898/0 (socket says 172.21.15.31:50898) 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e1d9b640 1 -- 172.21.15.31:0/3127846007 learned_addr learned my addr 172.21.15.31:0/3127846007 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e259c640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e4100e90 0x7f18e4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e2d9d640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e41029a0 0x7f18e419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e1d9b640 1 -- 172.21.15.31:0/3127846007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e4100e90 msgr2=0x7f18e4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e1d9b640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e4100e90 0x7f18e4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e1d9b640 1 -- 172.21.15.31:0/3127846007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e41029a0 msgr2=0x7f18e419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e1d9b640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e41029a0 0x7f18e419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e1d9b640 1 -- 172.21.15.31:0/3127846007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e419feb0 con 0x7f18e4101fe0 2024-04-25T13:17:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e1d9b640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e4101fe0 0x7f18e41987d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f18d800b450 tx=0x7f18d800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18d37fe640 1 -- 172.21.15.31:0/3127846007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8015020 con 0x7f18e4101fe0 2024-04-25T13:17:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18d37fe640 1 -- 172.21.15.31:0/3127846007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d8011d40 con 0x7f18e4101fe0 2024-04-25T13:17:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e898e640 1 -- 172.21.15.31:0/3127846007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e41a01a0 con 0x7f18e4101fe0 2024-04-25T13:17:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18d37fe640 1 -- 172.21.15.31:0/3127846007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d800f340 con 0x7f18e4101fe0 2024-04-25T13:17:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.433+0000 7f18e898e640 1 -- 172.21.15.31:0/3127846007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e41a06a0 con 0x7f18e4101fe0 2024-04-25T13:17:47.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.434+0000 7f18e898e640 1 -- 172.21.15.31:0/3127846007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e4108270 con 0x7f18e4101fe0 2024-04-25T13:17:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.435+0000 7f18d37fe640 1 -- 172.21.15.31:0/3127846007 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f18d8018020 con 0x7f18e4101fe0 2024-04-25T13:17:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.436+0000 7f18d37fe640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18cc0a3c40 0x7f18cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.436+0000 7f18d37fe640 1 -- 172.21.15.31:0/3127846007 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f18d80cfd20 con 0x7f18e4101fe0 2024-04-25T13:17:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.436+0000 7f18e259c640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18cc0a3c40 0x7f18cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.437+0000 7f18e259c640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18cc0a3c40 0x7f18cc0a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000c00 tx=0x7f18dc0314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:47.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.441+0000 7f18d37fe640 1 -- 172.21.15.31:0/3127846007 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18d801f020 con 0x7f18e4101fe0 2024-04-25T13:17:47.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.625+0000 7f18e898e640 1 -- 172.21.15.31:0/3127846007 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18e4062390 con 0x7f18cc0a3c40 2024-04-25T13:17:47.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.633+0000 7f18d37fe640 1 -- 172.21.15.31:0/3127846007 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258715 (secure 0 0 0) 0x7f18e4062390 con 0x7f18cc0a3c40 2024-04-25T13:17:47.637 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:47.638 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T13:17:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.635+0000 7f18e898e640 1 -- 172.21.15.31:0/3127846007 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18cc0a3c40 msgr2=0x7f18cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.636+0000 7f18e898e640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18cc0a3c40 0x7f18cc0a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000c00 tx=0x7f18dc0314a0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.636+0000 7f18e898e640 1 -- 172.21.15.31:0/3127846007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e4101fe0 msgr2=0x7f18e41987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.636+0000 7f18e898e640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e4101fe0 0x7f18e41987d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f18d800b450 tx=0x7f18d800b910 comp rx=0 tx=0).stop 2024-04-25T13:17:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.636+0000 7f18e898e640 1 -- 172.21.15.31:0/3127846007 shutdown_connections 2024-04-25T13:17:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.636+0000 7f18e898e640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18cc0a3c40 0x7f18cc0a6100 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.636+0000 7f18e898e640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18e41029a0 0x7f18e419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.636+0000 7f18e898e640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18e4101fe0 0x7f18e41987d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.636+0000 7f18e898e640 1 --2- 172.21.15.31:0/3127846007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e4100e90 0x7f18e4198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.636+0000 7f18e898e640 1 -- 172.21.15.31:0/3127846007 >> 172.21.15.31:0/3127846007 conn(0x7f18e40fc400 msgr2=0x7f18e4105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.636+0000 7f18e898e640 1 -- 172.21.15.31:0/3127846007 shutdown_connections 2024-04-25T13:17:47.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.636+0000 7f18e898e640 1 -- 172.21.15.31:0/3127846007 wait complete. 2024-04-25T13:17:47.648 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-04-25T13:17:45.801703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":140,"num_read_kb":123,"num_write":226,"num_write_kb":2952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176,"ondisk_log_size":176,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382780,"kb_used_data":12760,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559044,"statfs":{"total":1151940427776,"available":1151548461056,"internally_reserved":0,"allocated":13066240,"data_stored":7129308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001781"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817724+0000","last_change":"2024-04-25T13:17:40.825728+0000","last_active":"2024-04-25T13:17:41.817724+0000","last_peered":"2024-04-25T13:17:41.817724+0000","last_clean":"2024-04-25T13:17:41.817724+0000","last_became_active":"2024-04-25T13:17:40.824933+0000","last_became_peered":"2024-04-25T13:17:40.824933+0000","last_unstale":"2024-04-25T13:17:41.817724+0000","last_undegraded":"2024-04-25T13:17:41.817724+0000","last_fullsized":"2024-04-25T13:17:41.817724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:14.326614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.822754+0000","last_change":"2024-04-25T13:17:40.821148+0000","last_active":"2024-04-25T13:17:40.822754+0000","last_peered":"2024-04-25T13:17:40.822754+0000","last_clean":"2024-04-25T13:17:40.822754+0000","last_became_active":"2024-04-25T13:17:40.820244+0000","last_became_peered":"2024-04-25T13:17:40.820244+0000","last_unstale":"2024-04-25T13:17:40.822754+0000","last_undegraded":"2024-04-25T13:17:40.822754+0000","last_fullsized":"2024-04-25T13:17:40.822754+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:55.935937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.821110+0000","last_change":"2024-04-25T13:17:40.818729+0000","last_active":"2024-04-25T13:17:40.821110+0000","last_peered":"2024-04-25T13:17:40.821110+0000","last_clean":"2024-04-25T13:17:40.821110+0000","last_became_active":"2024-04-25T13:17:40.818351+0000","last_became_peered":"2024-04-25T13:17:40.818351+0000","last_unstale":"2024-04-25T13:17:40.821110+0000","last_undegraded":"2024-04-25T13:17:40.821110+0000","last_fullsized":"2024-04-25T13:17:40.821110+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:19.158406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.543414+0000","last_change":"2024-04-25T13:17:40.820932+0000","last_active":"2024-04-25T13:17:43.543414+0000","last_peered":"2024-04-25T13:17:43.543414+0000","last_clean":"2024-04-25T13:17:43.543414+0000","last_became_active":"2024-04-25T13:17:40.820525+0000","last_became_peered":"2024-04-25T13:17:40.820525+0000","last_unstale":"2024-04-25T13:17:43.543414+0000","last_undegraded":"2024-04-25T13:17:43.543414+0000","last_fullsized":"2024-04-25T13:17:43.543414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:12.077703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800540+0000","last_change":"2024-04-25T13:17:40.829959+0000","last_active":"2024-04-25T13:17:42.800540+0000","last_peered":"2024-04-25T13:17:42.800540+0000","last_clean":"2024-04-25T13:17:42.800540+0000","last_became_active":"2024-04-25T13:17:40.829594+0000","last_became_peered":"2024-04-25T13:17:40.829594+0000","last_unstale":"2024-04-25T13:17:42.800540+0000","last_undegraded":"2024-04-25T13:17:42.800540+0000","last_fullsized":"2024-04-25T13:17:42.800540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:55.719296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821448+0000","last_change":"2024-04-25T13:17:40.817436+0000","last_active":"2024-04-25T13:17:41.821448+0000","last_peered":"2024-04-25T13:17:41.821448+0000","last_clean":"2024-04-25T13:17:41.821448+0000","last_became_active":"2024-04-25T13:17:40.816763+0000","last_became_peered":"2024-04-25T13:17:40.816763+0000","last_unstale":"2024-04-25T13:17:41.821448+0000","last_undegraded":"2024-04-25T13:17:41.821448+0000","last_fullsized":"2024-04-25T13:17:41.821448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:09.505890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283501+0000","last_change":"2024-04-25T13:17:40.822968+0000","last_active":"2024-04-25T13:17:42.283501+0000","last_peered":"2024-04-25T13:17:42.283501+0000","last_clean":"2024-04-25T13:17:42.283501+0000","last_became_active":"2024-04-25T13:17:40.822600+0000","last_became_peered":"2024-04-25T13:17:40.822600+0000","last_unstale":"2024-04-25T13:17:42.283501+0000","last_undegraded":"2024-04-25T13:17:42.283501+0000","last_fullsized":"2024-04-25T13:17:42.283501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:08:32.276310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.809857+0000","last_change":"2024-04-25T13:17:40.823421+0000","last_active":"2024-04-25T13:17:42.809857+0000","last_peered":"2024-04-25T13:17:42.809857+0000","last_clean":"2024-04-25T13:17:42.809857+0000","last_became_active":"2024-04-25T13:17:40.823054+0000","last_became_peered":"2024-04-25T13:17:40.823054+0000","last_unstale":"2024-04-25T13:17:42.809857+0000","last_undegraded":"2024-04-25T13:17:42.809857+0000","last_fullsized":"2024-04-25T13:17:42.809857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:34.256763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.826603+0000","last_change":"2024-04-25T13:17:40.826105+0000","last_active":"2024-04-25T13:17:40.826603+0000","last_peered":"2024-04-25T13:17:40.826603+0000","last_clean":"2024-04-25T13:17:40.826603+0000","last_became_active":"2024-04-25T13:17:40.825763+0000","last_became_peered":"2024-04-25T13:17:40.825763+0000","last_unstale":"2024-04-25T13:17:40.826603+0000","last_undegraded":"2024-04-25T13:17:40.826603+0000","last_fullsized":"2024-04-25T13:17:40.826603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:41.417672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821749+0000","last_change":"2024-04-25T13:17:40.817356+0000","last_active":"2024-04-25T13:17:41.821749+0000","last_peered":"2024-04-25T13:17:41.821749+0000","last_clean":"2024-04-25T13:17:41.821749+0000","last_became_active":"2024-04-25T13:17:40.816906+0000","last_became_peered":"2024-04-25T13:17:40.816906+0000","last_unstale":"2024-04-25T13:17:41.821749+0000","last_undegraded":"2024-04-25T13:17:41.821749+0000","last_fullsized":"2024-04-25T13:17:41.821749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283927+0000","last_change":"2024-04-25T13:17:40.829445+0000","last_active":"2024-04-25T13:17:42.283927+0000","last_peered":"2024-04-25T13:17:42.283927+0000","last_clean":"2024-04-25T13:17:42.283927+0000","last_became_active":"2024-04-25T13:17:40.826709+0000","last_became_peered":"2024-04-25T13:17:40.826709+0000","last_unstale":"2024-04-25T13:17:42.283927+0000","last_undegraded":"2024-04-25T13:17:42.283927+0000","last_fullsized":"2024-04-25T13:17:42.283927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:43.044192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.543222+0000","last_change":"2024-04-25T13:17:40.827486+0000","last_active":"2024-04-25T13:17:43.543222+0000","last_peered":"2024-04-25T13:17:43.543222+0000","last_clean":"2024-04-25T13:17:43.543222+0000","last_became_active":"2024-04-25T13:17:40.827070+0000","last_became_peered":"2024-04-25T13:17:40.827070+0000","last_unstale":"2024-04-25T13:17:43.543222+0000","last_undegraded":"2024-04-25T13:17:43.543222+0000","last_fullsized":"2024-04-25T13:17:43.543222+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:39.564567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311636+0000","last_change":"2024-04-25T13:17:40.826645+0000","last_active":"2024-04-25T13:17:43.311636+0000","last_peered":"2024-04-25T13:17:43.311636+0000","last_clean":"2024-04-25T13:17:43.311636+0000","last_became_active":"2024-04-25T13:17:40.826314+0000","last_became_peered":"2024-04-25T13:17:40.826314+0000","last_unstale":"2024-04-25T13:17:43.311636+0000","last_undegraded":"2024-04-25T13:17:43.311636+0000","last_fullsized":"2024-04-25T13:17:43.311636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:59.600451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.817079+0000","last_change":"2024-04-25T13:17:40.814918+0000","last_active":"2024-04-25T13:17:40.817079+0000","last_peered":"2024-04-25T13:17:40.817079+0000","last_clean":"2024-04-25T13:17:40.817079+0000","last_became_active":"2024-04-25T13:17:40.814573+0000","last_became_peered":"2024-04-25T13:17:40.814573+0000","last_unstale":"2024-04-25T13:17:40.817079+0000","last_undegraded":"2024-04-25T13:17:40.817079+0000","last_fullsized":"2024-04-25T13:17:40.817079+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:20.601194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.826295+0000","last_change":"2024-04-25T13:17:40.825481+0000","last_active":"2024-04-25T13:17:40.826295+0000","last_peered":"2024-04-25T13:17:40.826295+0000","last_clean":"2024-04-25T13:17:40.826295+0000","last_became_active":"2024-04-25T13:17:40.823726+0000","last_became_peered":"2024-04-25T13:17:40.823726+0000","last_unstale":"2024-04-25T13:17:40.826295+0000","last_undegraded":"2024-04-25T13:17:40.826295+0000","last_fullsized":"2024-04-25T13:17:40.826295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:28.876404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.828575+0000","last_change":"2024-04-25T13:17:40.827561+0000","last_active":"2024-04-25T13:17:40.828575+0000","last_peered":"2024-04-25T13:17:40.828575+0000","last_clean":"2024-04-25T13:17:40.828575+0000","last_became_active":"2024-04-25T13:17:40.827100+0000","last_became_peered":"2024-04-25T13:17:40.827100+0000","last_unstale":"2024-04-25T13:17:40.828575+0000","last_undegraded":"2024-04-25T13:17:40.828575+0000","last_fullsized":"2024-04-25T13:17:40.828575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:22.607745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817824+0000","last_change":"2024-04-25T13:17:40.826017+0000","last_active":"2024-04-25T13:17:41.817824+0000","last_peered":"2024-04-25T13:17:41.817824+0000","last_clean":"2024-04-25T13:17:41.817824+0000","last_became_active":"2024-04-25T13:17:40.825429+0000","last_became_peered":"2024-04-25T13:17:40.825429+0000","last_unstale":"2024-04-25T13:17:41.817824+0000","last_undegraded":"2024-04-25T13:17:41.817824+0000","last_fullsized":"2024-04-25T13:17:41.817824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:58.788260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.816491+0000","last_change":"2024-04-25T13:17:40.830115+0000","last_active":"2024-04-25T13:17:41.816491+0000","last_peered":"2024-04-25T13:17:41.816491+0000","last_clean":"2024-04-25T13:17:41.816491+0000","last_became_active":"2024-04-25T13:17:40.829820+0000","last_became_peered":"2024-04-25T13:17:40.829820+0000","last_unstale":"2024-04-25T13:17:41.816491+0000","last_undegraded":"2024-04-25T13:17:41.816491+0000","last_fullsized":"2024-04-25T13:17:41.816491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:35.051183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.825494+0000","last_change":"2024-04-25T13:17:40.824689+0000","last_active":"2024-04-25T13:17:40.825494+0000","last_peered":"2024-04-25T13:17:40.825494+0000","last_clean":"2024-04-25T13:17:40.825494+0000","last_became_active":"2024-04-25T13:17:40.823878+0000","last_became_peered":"2024-04-25T13:17:40.823878+0000","last_unstale":"2024-04-25T13:17:40.825494+0000","last_undegraded":"2024-04-25T13:17:40.825494+0000","last_fullsized":"2024-04-25T13:17:40.825494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:08.928531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.816449+0000","last_change":"2024-04-25T13:17:39.812816+0000","last_active":"2024-04-25T13:17:41.816449+0000","last_peered":"2024-04-25T13:17:41.816449+0000","last_clean":"2024-04-25T13:17:41.816449+0000","last_became_active":"2024-04-25T13:17:39.812483+0000","last_became_peered":"2024-04-25T13:17:39.812483+0000","last_unstale":"2024-04-25T13:17:41.816449+0000","last_undegraded":"2024-04-25T13:17:41.816449+0000","last_fullsized":"2024-04-25T13:17:41.816449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:14.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810295+0000","last_change":"2024-04-25T13:17:40.824764+0000","last_active":"2024-04-25T13:17:42.810295+0000","last_peered":"2024-04-25T13:17:42.810295+0000","last_clean":"2024-04-25T13:17:42.810295+0000","last_became_active":"2024-04-25T13:17:40.824337+0000","last_became_peered":"2024-04-25T13:17:40.824337+0000","last_unstale":"2024-04-25T13:17:42.810295+0000","last_undegraded":"2024-04-25T13:17:42.810295+0000","last_fullsized":"2024-04-25T13:17:42.810295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:37.125246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810544+0000","last_change":"2024-04-25T13:17:39.819820+0000","last_active":"2024-04-25T13:17:42.810544+0000","last_peered":"2024-04-25T13:17:42.810544+0000","last_clean":"2024-04-25T13:17:42.810544+0000","last_became_active":"2024-04-25T13:17:39.819446+0000","last_became_peered":"2024-04-25T13:17:39.819446+0000","last_unstale":"2024-04-25T13:17:42.810544+0000","last_undegraded":"2024-04-25T13:17:42.810544+0000","last_fullsized":"2024-04-25T13:17:42.810544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:36.902011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.822785+0000","last_change":"2024-04-25T13:17:40.821183+0000","last_active":"2024-04-25T13:17:40.822785+0000","last_peered":"2024-04-25T13:17:40.822785+0000","last_clean":"2024-04-25T13:17:40.822785+0000","last_became_active":"2024-04-25T13:17:40.820789+0000","last_became_peered":"2024-04-25T13:17:40.820789+0000","last_unstale":"2024-04-25T13:17:40.822785+0000","last_undegraded":"2024-04-25T13:17:40.822785+0000","last_fullsized":"2024-04-25T13:17:40.822785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:36.500295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800656+0000","last_change":"2024-04-25T13:17:39.813132+0000","last_active":"2024-04-25T13:17:42.800656+0000","last_peered":"2024-04-25T13:17:42.800656+0000","last_clean":"2024-04-25T13:17:42.800656+0000","last_became_active":"2024-04-25T13:17:39.812837+0000","last_became_peered":"2024-04-25T13:17:39.812837+0000","last_unstale":"2024-04-25T13:17:42.800656+0000","last_undegraded":"2024-04-25T13:17:42.800656+0000","last_fullsized":"2024-04-25T13:17:42.800656+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:55.342376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.824935+0000","last_change":"2024-04-25T13:17:40.824143+0000","last_active":"2024-04-25T13:17:40.824935+0000","last_peered":"2024-04-25T13:17:40.824935+0000","last_clean":"2024-04-25T13:17:40.824935+0000","last_became_active":"2024-04-25T13:17:40.823778+0000","last_became_peered":"2024-04-25T13:17:40.823778+0000","last_unstale":"2024-04-25T13:17:40.824935+0000","last_undegraded":"2024-04-25T13:17:40.824935+0000","last_fullsized":"2024-04-25T13:17:40.824935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:17.554632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800644+0000","last_change":"2024-04-25T13:17:39.817743+0000","last_active":"2024-04-25T13:17:42.800644+0000","last_peered":"2024-04-25T13:17:42.800644+0000","last_clean":"2024-04-25T13:17:42.800644+0000","last_became_active":"2024-04-25T13:17:39.817392+0000","last_became_peered":"2024-04-25T13:17:39.817392+0000","last_unstale":"2024-04-25T13:17:42.800644+0000","last_undegraded":"2024-04-25T13:17:42.800644+0000","last_fullsized":"2024-04-25T13:17:42.800644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.400644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.832661+0000","last_change":"2024-04-25T13:17:40.829895+0000","last_active":"2024-04-25T13:17:40.832661+0000","last_peered":"2024-04-25T13:17:40.832661+0000","last_clean":"2024-04-25T13:17:40.832661+0000","last_became_active":"2024-04-25T13:17:40.829300+0000","last_became_peered":"2024-04-25T13:17:40.829300+0000","last_unstale":"2024-04-25T13:17:40.832661+0000","last_undegraded":"2024-04-25T13:17:40.832661+0000","last_fullsized":"2024-04-25T13:17:40.832661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:54.300946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.831939+0000","last_change":"2024-04-25T13:17:39.813231+0000","last_active":"2024-04-25T13:17:40.831939+0000","last_peered":"2024-04-25T13:17:40.831939+0000","last_clean":"2024-04-25T13:17:40.831939+0000","last_became_active":"2024-04-25T13:17:39.812441+0000","last_became_peered":"2024-04-25T13:17:39.812441+0000","last_unstale":"2024-04-25T13:17:40.831939+0000","last_undegraded":"2024-04-25T13:17:40.831939+0000","last_fullsized":"2024-04-25T13:17:40.831939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:17.578986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800414+0000","last_change":"2024-04-25T13:17:40.823309+0000","last_active":"2024-04-25T13:17:42.800414+0000","last_peered":"2024-04-25T13:17:42.800414+0000","last_clean":"2024-04-25T13:17:42.800414+0000","last_became_active":"2024-04-25T13:17:40.822909+0000","last_became_peered":"2024-04-25T13:17:40.822909+0000","last_unstale":"2024-04-25T13:17:42.800414+0000","last_undegraded":"2024-04-25T13:17:42.800414+0000","last_fullsized":"2024-04-25T13:17:42.800414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:07.124306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821197+0000","last_change":"2024-04-25T13:17:39.809888+0000","last_active":"2024-04-25T13:17:41.821197+0000","last_peered":"2024-04-25T13:17:41.821197+0000","last_clean":"2024-04-25T13:17:41.821197+0000","last_became_active":"2024-04-25T13:17:39.809519+0000","last_became_peered":"2024-04-25T13:17:39.809519+0000","last_unstale":"2024-04-25T13:17:41.821197+0000","last_undegraded":"2024-04-25T13:17:41.821197+0000","last_fullsized":"2024-04-25T13:17:41.821197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:43.306668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.832105+0000","last_change":"2024-04-25T13:17:40.828666+0000","last_active":"2024-04-25T13:17:40.832105+0000","last_peered":"2024-04-25T13:17:40.832105+0000","last_clean":"2024-04-25T13:17:40.832105+0000","last_became_active":"2024-04-25T13:17:40.828332+0000","last_became_peered":"2024-04-25T13:17:40.828332+0000","last_unstale":"2024-04-25T13:17:40.832105+0000","last_undegraded":"2024-04-25T13:17:40.832105+0000","last_fullsized":"2024-04-25T13:17:40.832105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:24.077116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311412+0000","last_change":"2024-04-25T13:17:39.811231+0000","last_active":"2024-04-25T13:17:43.311412+0000","last_peered":"2024-04-25T13:17:43.311412+0000","last_clean":"2024-04-25T13:17:43.311412+0000","last_became_active":"2024-04-25T13:17:39.810762+0000","last_became_peered":"2024-04-25T13:17:39.810762+0000","last_unstale":"2024-04-25T13:17:43.311412+0000","last_undegraded":"2024-04-25T13:17:43.311412+0000","last_fullsized":"2024-04-25T13:17:43.311412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:15.970042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810115+0000","last_change":"2024-04-25T13:17:40.828074+0000","last_active":"2024-04-25T13:17:42.810115+0000","last_peered":"2024-04-25T13:17:42.810115+0000","last_clean":"2024-04-25T13:17:42.810115+0000","last_became_active":"2024-04-25T13:17:40.827702+0000","last_became_peered":"2024-04-25T13:17:40.827702+0000","last_unstale":"2024-04-25T13:17:42.810115+0000","last_undegraded":"2024-04-25T13:17:42.810115+0000","last_fullsized":"2024-04-25T13:17:42.810115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:23.408908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.815210+0000","last_change":"2024-04-25T13:17:39.812746+0000","last_active":"2024-04-25T13:17:40.815210+0000","last_peered":"2024-04-25T13:17:40.815210+0000","last_clean":"2024-04-25T13:17:40.815210+0000","last_became_active":"2024-04-25T13:17:39.812163+0000","last_became_peered":"2024-04-25T13:17:39.812163+0000","last_unstale":"2024-04-25T13:17:40.815210+0000","last_undegraded":"2024-04-25T13:17:40.815210+0000","last_fullsized":"2024-04-25T13:17:40.815210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:20.410328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.816285+0000","last_change":"2024-04-25T13:17:40.829512+0000","last_active":"2024-04-25T13:17:41.816285+0000","last_peered":"2024-04-25T13:17:41.816285+0000","last_clean":"2024-04-25T13:17:41.816285+0000","last_became_active":"2024-04-25T13:17:40.829084+0000","last_became_peered":"2024-04-25T13:17:40.829084+0000","last_unstale":"2024-04-25T13:17:41.816285+0000","last_undegraded":"2024-04-25T13:17:41.816285+0000","last_fullsized":"2024-04-25T13:17:41.816285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:55.473018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821500+0000","last_change":"2024-04-25T13:17:39.810766+0000","last_active":"2024-04-25T13:17:41.821500+0000","last_peered":"2024-04-25T13:17:41.821500+0000","last_clean":"2024-04-25T13:17:41.821500+0000","last_became_active":"2024-04-25T13:17:39.810444+0000","last_became_peered":"2024-04-25T13:17:39.810444+0000","last_unstale":"2024-04-25T13:17:41.821500+0000","last_undegraded":"2024-04-25T13:17:41.821500+0000","last_fullsized":"2024-04-25T13:17:41.821500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:33.470819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.826336+0000","last_change":"2024-04-25T13:17:40.825574+0000","last_active":"2024-04-25T13:17:40.826336+0000","last_peered":"2024-04-25T13:17:40.826336+0000","last_clean":"2024-04-25T13:17:40.826336+0000","last_became_active":"2024-04-25T13:17:40.825220+0000","last_became_peered":"2024-04-25T13:17:40.825220+0000","last_unstale":"2024-04-25T13:17:40.826336+0000","last_undegraded":"2024-04-25T13:17:40.826336+0000","last_fullsized":"2024-04-25T13:17:40.826336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:10.545369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283795+0000","last_change":"2024-04-25T13:17:39.816040+0000","last_active":"2024-04-25T13:17:42.283795+0000","last_peered":"2024-04-25T13:17:42.283795+0000","last_clean":"2024-04-25T13:17:42.283795+0000","last_became_active":"2024-04-25T13:17:39.815746+0000","last_became_peered":"2024-04-25T13:17:39.815746+0000","last_unstale":"2024-04-25T13:17:42.283795+0000","last_undegraded":"2024-04-25T13:17:42.283795+0000","last_fullsized":"2024-04-25T13:17:42.283795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:12.068992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.822886+0000","last_change":"2024-04-25T13:17:40.821926+0000","last_active":"2024-04-25T13:17:40.822886+0000","last_peered":"2024-04-25T13:17:40.822886+0000","last_clean":"2024-04-25T13:17:40.822886+0000","last_became_active":"2024-04-25T13:17:40.821280+0000","last_became_peered":"2024-04-25T13:17:40.821280+0000","last_unstale":"2024-04-25T13:17:40.822886+0000","last_undegraded":"2024-04-25T13:17:40.822886+0000","last_fullsized":"2024-04-25T13:17:40.822886+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:43.980843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.838130+0000","last_change":"2024-04-25T13:17:39.819113+0000","last_active":"2024-04-25T13:17:40.838130+0000","last_peered":"2024-04-25T13:17:40.838130+0000","last_clean":"2024-04-25T13:17:40.838130+0000","last_became_active":"2024-04-25T13:17:39.818749+0000","last_became_peered":"2024-04-25T13:17:39.818749+0000","last_unstale":"2024-04-25T13:17:40.838130+0000","last_undegraded":"2024-04-25T13:17:40.838130+0000","last_fullsized":"2024-04-25T13:17:40.838130+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:32.119616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.822214+0000","last_change":"2024-04-25T13:17:40.821066+0000","last_active":"2024-04-25T13:17:40.822214+0000","last_peered":"2024-04-25T13:17:40.822214+0000","last_clean":"2024-04-25T13:17:40.822214+0000","last_became_active":"2024-04-25T13:17:40.820674+0000","last_became_peered":"2024-04-25T13:17:40.820674+0000","last_unstale":"2024-04-25T13:17:40.822214+0000","last_undegraded":"2024-04-25T13:17:40.822214+0000","last_fullsized":"2024-04-25T13:17:40.822214+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:32.274461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821501+0000","last_change":"2024-04-25T13:17:39.811131+0000","last_active":"2024-04-25T13:17:41.821501+0000","last_peered":"2024-04-25T13:17:41.821501+0000","last_clean":"2024-04-25T13:17:41.821501+0000","last_became_active":"2024-04-25T13:17:39.810791+0000","last_became_peered":"2024-04-25T13:17:39.810791+0000","last_unstale":"2024-04-25T13:17:41.821501+0000","last_undegraded":"2024-04-25T13:17:41.821501+0000","last_fullsized":"2024-04-25T13:17:41.821501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:43.964183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800602+0000","last_change":"2024-04-25T13:17:40.818077+0000","last_active":"2024-04-25T13:17:42.800602+0000","last_peered":"2024-04-25T13:17:42.800602+0000","last_clean":"2024-04-25T13:17:42.800602+0000","last_became_active":"2024-04-25T13:17:40.817757+0000","last_became_peered":"2024-04-25T13:17:40.817757+0000","last_unstale":"2024-04-25T13:17:42.800602+0000","last_undegraded":"2024-04-25T13:17:42.800602+0000","last_fullsized":"2024-04-25T13:17:42.800602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:17.526603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.830107+0000","last_change":"2024-04-25T13:17:39.809111+0000","last_active":"2024-04-25T13:17:40.830107+0000","last_peered":"2024-04-25T13:17:40.830107+0000","last_clean":"2024-04-25T13:17:40.830107+0000","last_became_active":"2024-04-25T13:17:39.808785+0000","last_became_peered":"2024-04-25T13:17:39.808785+0000","last_unstale":"2024-04-25T13:17:40.830107+0000","last_undegraded":"2024-04-25T13:17:40.830107+0000","last_fullsized":"2024-04-25T13:17:40.830107+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:41.549940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311564+0000","last_change":"2024-04-25T13:17:40.835646+0000","last_active":"2024-04-25T13:17:43.311564+0000","last_peered":"2024-04-25T13:17:43.311564+0000","last_clean":"2024-04-25T13:17:43.311564+0000","last_became_active":"2024-04-25T13:17:40.835267+0000","last_became_peered":"2024-04-25T13:17:40.835267+0000","last_unstale":"2024-04-25T13:17:43.311564+0000","last_undegraded":"2024-04-25T13:17:43.311564+0000","last_fullsized":"2024-04-25T13:17:43.311564+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:05.811651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.810881+0000","last_change":"2024-04-25T13:17:39.813006+0000","last_active":"2024-04-25T13:17:40.810881+0000","last_peered":"2024-04-25T13:17:40.810881+0000","last_clean":"2024-04-25T13:17:40.810881+0000","last_became_active":"2024-04-25T13:17:39.812635+0000","last_became_peered":"2024-04-25T13:17:39.812635+0000","last_unstale":"2024-04-25T13:17:40.810881+0000","last_undegraded":"2024-04-25T13:17:40.810881+0000","last_fullsized":"2024-04-25T13:17:40.810881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.922568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283851+0000","last_change":"2024-04-25T13:17:40.833365+0000","last_active":"2024-04-25T13:17:42.283851+0000","last_peered":"2024-04-25T13:17:42.283851+0000","last_clean":"2024-04-25T13:17:42.283851+0000","last_became_active":"2024-04-25T13:17:40.832927+0000","last_became_peered":"2024-04-25T13:17:40.832927+0000","last_unstale":"2024-04-25T13:17:42.283851+0000","last_undegraded":"2024-04-25T13:17:42.283851+0000","last_fullsized":"2024-04-25T13:17:42.283851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.622153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.542809+0000","last_change":"2024-04-25T13:17:39.810983+0000","last_active":"2024-04-25T13:17:43.542809+0000","last_peered":"2024-04-25T13:17:43.542809+0000","last_clean":"2024-04-25T13:17:43.542809+0000","last_became_active":"2024-04-25T13:17:39.810366+0000","last_became_peered":"2024-04-25T13:17:39.810366+0000","last_unstale":"2024-04-25T13:17:43.542809+0000","last_undegraded":"2024-04-25T13:17:43.542809+0000","last_fullsized":"2024-04-25T13:17:43.542809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:33.636924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283777+0000","last_change":"2024-04-25T13:17:40.818301+0000","last_active":"2024-04-25T13:17:42.283777+0000","last_peered":"2024-04-25T13:17:42.283777+0000","last_clean":"2024-04-25T13:17:42.283777+0000","last_became_active":"2024-04-25T13:17:40.817903+0000","last_became_peered":"2024-04-25T13:17:40.817903+0000","last_unstale":"2024-04-25T13:17:42.283777+0000","last_undegraded":"2024-04-25T13:17:42.283777+0000","last_fullsized":"2024-04-25T13:17:42.283777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:40.913771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800097+0000","last_change":"2024-04-25T13:17:39.820411+0000","last_active":"2024-04-25T13:17:42.800097+0000","last_peered":"2024-04-25T13:17:42.800097+0000","last_clean":"2024-04-25T13:17:42.800097+0000","last_became_active":"2024-04-25T13:17:39.820051+0000","last_became_peered":"2024-04-25T13:17:39.820051+0000","last_unstale":"2024-04-25T13:17:42.800097+0000","last_undegraded":"2024-04-25T13:17:42.800097+0000","last_fullsized":"2024-04-25T13:17:42.800097+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:08.931024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.832139+0000","last_change":"2024-04-25T13:17:40.828743+0000","last_active":"2024-04-25T13:17:40.832139+0000","last_peered":"2024-04-25T13:17:40.832139+0000","last_clean":"2024-04-25T13:17:40.832139+0000","last_became_active":"2024-04-25T13:17:40.828397+0000","last_became_peered":"2024-04-25T13:17:40.828397+0000","last_unstale":"2024-04-25T13:17:40.832139+0000","last_undegraded":"2024-04-25T13:17:40.832139+0000","last_fullsized":"2024-04-25T13:17:40.832139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:48.099806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800606+0000","last_change":"2024-04-25T13:17:39.812160+0000","last_active":"2024-04-25T13:17:42.800606+0000","last_peered":"2024-04-25T13:17:42.800606+0000","last_clean":"2024-04-25T13:17:42.800606+0000","last_became_active":"2024-04-25T13:17:39.811846+0000","last_became_peered":"2024-04-25T13:17:39.811846+0000","last_unstale":"2024-04-25T13:17:42.800606+0000","last_undegraded":"2024-04-25T13:17:42.800606+0000","last_fullsized":"2024-04-25T13:17:42.800606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:20.146245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817653+0000","last_change":"2024-04-25T13:17:40.819439+0000","last_active":"2024-04-25T13:17:41.817653+0000","last_peered":"2024-04-25T13:17:41.817653+0000","last_clean":"2024-04-25T13:17:41.817653+0000","last_became_active":"2024-04-25T13:17:40.819003+0000","last_became_peered":"2024-04-25T13:17:40.819003+0000","last_unstale":"2024-04-25T13:17:41.817653+0000","last_undegraded":"2024-04-25T13:17:41.817653+0000","last_fullsized":"2024-04-25T13:17:41.817653+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:43.405033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.816466+0000","last_change":"2024-04-25T13:17:39.819540+0000","last_active":"2024-04-25T13:17:41.816466+0000","last_peered":"2024-04-25T13:17:41.816466+0000","last_clean":"2024-04-25T13:17:41.816466+0000","last_became_active":"2024-04-25T13:17:39.818942+0000","last_became_peered":"2024-04-25T13:17:39.818942+0000","last_unstale":"2024-04-25T13:17:41.816466+0000","last_undegraded":"2024-04-25T13:17:41.816466+0000","last_fullsized":"2024-04-25T13:17:41.816466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:51.300087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821688+0000","last_change":"2024-04-25T13:17:40.818872+0000","last_active":"2024-04-25T13:17:41.821688+0000","last_peered":"2024-04-25T13:17:41.821688+0000","last_clean":"2024-04-25T13:17:41.821688+0000","last_became_active":"2024-04-25T13:17:40.818522+0000","last_became_peered":"2024-04-25T13:17:40.818522+0000","last_unstale":"2024-04-25T13:17:41.821688+0000","last_undegraded":"2024-04-25T13:17:41.821688+0000","last_fullsized":"2024-04-25T13:17:41.821688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.928782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283549+0000","last_change":"2024-04-25T13:17:39.815555+0000","last_active":"2024-04-25T13:17:42.283549+0000","last_peered":"2024-04-25T13:17:42.283549+0000","last_clean":"2024-04-25T13:17:42.283549+0000","last_became_active":"2024-04-25T13:17:39.815249+0000","last_became_peered":"2024-04-25T13:17:39.815249+0000","last_unstale":"2024-04-25T13:17:42.283549+0000","last_undegraded":"2024-04-25T13:17:42.283549+0000","last_fullsized":"2024-04-25T13:17:42.283549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:40.723562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821493+0000","last_change":"2024-04-25T13:17:40.821643+0000","last_active":"2024-04-25T13:17:41.821493+0000","last_peered":"2024-04-25T13:17:41.821493+0000","last_clean":"2024-04-25T13:17:41.821493+0000","last_became_active":"2024-04-25T13:17:40.820715+0000","last_became_peered":"2024-04-25T13:17:40.820715+0000","last_unstale":"2024-04-25T13:17:41.821493+0000","last_undegraded":"2024-04-25T13:17:41.821493+0000","last_fullsized":"2024-04-25T13:17:41.821493+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:54.046030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.834347+0000","last_change":"2024-04-25T13:17:40.830693+0000","last_active":"2024-04-25T13:17:40.834347+0000","last_peered":"2024-04-25T13:17:40.834347+0000","last_clean":"2024-04-25T13:17:40.834347+0000","last_became_active":"2024-04-25T13:17:40.830298+0000","last_became_peered":"2024-04-25T13:17:40.830298+0000","last_unstale":"2024-04-25T13:17:40.834347+0000","last_undegraded":"2024-04-25T13:17:40.834347+0000","last_fullsized":"2024-04-25T13:17:40.834347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:22.324236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311284+0000","last_change":"2024-04-25T13:17:39.820658+0000","last_active":"2024-04-25T13:17:43.311284+0000","last_peered":"2024-04-25T13:17:43.311284+0000","last_clean":"2024-04-25T13:17:43.311284+0000","last_became_active":"2024-04-25T13:17:39.820122+0000","last_became_peered":"2024-04-25T13:17:39.820122+0000","last_unstale":"2024-04-25T13:17:43.311284+0000","last_undegraded":"2024-04-25T13:17:43.311284+0000","last_fullsized":"2024-04-25T13:17:43.311284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:04.655982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810232+0000","last_change":"2024-04-25T13:17:40.827220+0000","last_active":"2024-04-25T13:17:42.810232+0000","last_peered":"2024-04-25T13:17:42.810232+0000","last_clean":"2024-04-25T13:17:42.810232+0000","last_became_active":"2024-04-25T13:17:40.826517+0000","last_became_peered":"2024-04-25T13:17:40.826517+0000","last_unstale":"2024-04-25T13:17:42.810232+0000","last_undegraded":"2024-04-25T13:17:42.810232+0000","last_fullsized":"2024-04-25T13:17:42.810232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:30.405644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.542955+0000","last_change":"2024-04-25T13:17:39.813893+0000","last_active":"2024-04-25T13:17:43.542955+0000","last_peered":"2024-04-25T13:17:43.542955+0000","last_clean":"2024-04-25T13:17:43.542955+0000","last_became_active":"2024-04-25T13:17:39.812939+0000","last_became_peered":"2024-04-25T13:17:39.812939+0000","last_unstale":"2024-04-25T13:17:43.542955+0000","last_undegraded":"2024-04-25T13:17:43.542955+0000","last_fullsized":"2024-04-25T13:17:43.542955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.033913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283324+0000","last_change":"2024-04-25T13:16:30.734601+0000","last_active":"2024-04-25T13:17:42.283324+0000","last_peered":"2024-04-25T13:17:42.283324+0000","last_clean":"2024-04-25T13:17:42.283324+0000","last_became_active":"2024-04-25T13:16:30.733987+0000","last_became_peered":"2024-04-25T13:16:30.733987+0000","last_unstale":"2024-04-25T13:17:42.283324+0000","last_undegraded":"2024-04-25T13:17:42.283324+0000","last_fullsized":"2024-04-25T13:17:42.283324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:13:02.186391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:13:02.186391+0000","last_clean_scrub_stamp":"2024-04-25T13:13:02.186391+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:41.669205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311424+0000","last_change":"2024-04-25T13:17:40.821671+0000","last_active":"2024-04-25T13:17:43.311424+0000","last_peered":"2024-04-25T13:17:43.311424+0000","last_clean":"2024-04-25T13:17:43.311424+0000","last_became_active":"2024-04-25T13:17:40.821333+0000","last_became_peered":"2024-04-25T13:17:40.821333+0000","last_unstale":"2024-04-25T13:17:43.311424+0000","last_undegraded":"2024-04-25T13:17:43.311424+0000","last_fullsized":"2024-04-25T13:17:43.311424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:50.645335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283939+0000","last_change":"2024-04-25T13:17:39.809530+0000","last_active":"2024-04-25T13:17:42.283939+0000","last_peered":"2024-04-25T13:17:42.283939+0000","last_clean":"2024-04-25T13:17:42.283939+0000","last_became_active":"2024-04-25T13:17:39.809198+0000","last_became_peered":"2024-04-25T13:17:39.809198+0000","last_unstale":"2024-04-25T13:17:42.283939+0000","last_undegraded":"2024-04-25T13:17:42.283939+0000","last_fullsized":"2024-04-25T13:17:42.283939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:34.397912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800427+0000","last_change":"2024-04-25T13:17:39.819607+0000","last_active":"2024-04-25T13:17:42.800427+0000","last_peered":"2024-04-25T13:17:42.800427+0000","last_clean":"2024-04-25T13:17:42.800427+0000","last_became_active":"2024-04-25T13:17:39.819273+0000","last_became_peered":"2024-04-25T13:17:39.819273+0000","last_unstale":"2024-04-25T13:17:42.800427+0000","last_undegraded":"2024-04-25T13:17:42.800427+0000","last_fullsized":"2024-04-25T13:17:42.800427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:39.414085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311723+0000","last_change":"2024-04-25T13:17:40.824554+0000","last_active":"2024-04-25T13:17:43.311723+0000","last_peered":"2024-04-25T13:17:43.311723+0000","last_clean":"2024-04-25T13:17:43.311723+0000","last_became_active":"2024-04-25T13:17:40.824249+0000","last_became_peered":"2024-04-25T13:17:40.824249+0000","last_unstale":"2024-04-25T13:17:43.311723+0000","last_undegraded":"2024-04-25T13:17:43.311723+0000","last_fullsized":"2024-04-25T13:17:43.311723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:17.282312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.816510+0000","last_change":"2024-04-25T13:17:39.820095+0000","last_active":"2024-04-25T13:17:41.816510+0000","last_peered":"2024-04-25T13:17:41.816510+0000","last_clean":"2024-04-25T13:17:41.816510+0000","last_became_active":"2024-04-25T13:17:39.819774+0000","last_became_peered":"2024-04-25T13:17:39.819774+0000","last_unstale":"2024-04-25T13:17:41.816510+0000","last_undegraded":"2024-04-25T13:17:41.816510+0000","last_fullsized":"2024-04-25T13:17:41.816510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:09.154958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810602+0000","last_change":"2024-04-25T13:17:40.830202+0000","last_active":"2024-04-25T13:17:42.810602+0000","last_peered":"2024-04-25T13:17:42.810602+0000","last_clean":"2024-04-25T13:17:42.810602+0000","last_became_active":"2024-04-25T13:17:40.829814+0000","last_became_peered":"2024-04-25T13:17:40.829814+0000","last_unstale":"2024-04-25T13:17:42.810602+0000","last_undegraded":"2024-04-25T13:17:42.810602+0000","last_fullsized":"2024-04-25T13:17:42.810602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:05.492378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.543471+0000","last_change":"2024-04-25T13:17:39.814680+0000","last_active":"2024-04-25T13:17:43.543471+0000","last_peered":"2024-04-25T13:17:43.543471+0000","last_clean":"2024-04-25T13:17:43.543471+0000","last_became_active":"2024-04-25T13:17:39.814324+0000","last_became_peered":"2024-04-25T13:17:39.814324+0000","last_unstale":"2024-04-25T13:17:43.543471+0000","last_undegraded":"2024-04-25T13:17:43.543471+0000","last_fullsized":"2024-04-25T13:17:43.543471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:42.818060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.822506+0000","last_change":"2024-04-25T13:17:40.821020+0000","last_active":"2024-04-25T13:17:40.822506+0000","last_peered":"2024-04-25T13:17:40.822506+0000","last_clean":"2024-04-25T13:17:40.822506+0000","last_became_active":"2024-04-25T13:17:40.820664+0000","last_became_peered":"2024-04-25T13:17:40.820664+0000","last_unstale":"2024-04-25T13:17:40.822506+0000","last_undegraded":"2024-04-25T13:17:40.822506+0000","last_fullsized":"2024-04-25T13:17:40.822506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:50.224578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.807096+0000","last_change":"2024-04-25T13:17:39.820908+0000","last_active":"2024-04-25T13:17:40.807096+0000","last_peered":"2024-04-25T13:17:40.807096+0000","last_clean":"2024-04-25T13:17:40.807096+0000","last_became_active":"2024-04-25T13:17:39.820612+0000","last_became_peered":"2024-04-25T13:17:39.820612+0000","last_unstale":"2024-04-25T13:17:40.807096+0000","last_undegraded":"2024-04-25T13:17:40.807096+0000","last_fullsized":"2024-04-25T13:17:40.807096+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:15.170079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800250+0000","last_change":"2024-04-25T13:17:40.817232+0000","last_active":"2024-04-25T13:17:42.800250+0000","last_peered":"2024-04-25T13:17:42.800250+0000","last_clean":"2024-04-25T13:17:42.800250+0000","last_became_active":"2024-04-25T13:17:40.816895+0000","last_became_peered":"2024-04-25T13:17:40.816895+0000","last_unstale":"2024-04-25T13:17:42.800250+0000","last_undegraded":"2024-04-25T13:17:42.800250+0000","last_fullsized":"2024-04-25T13:17:42.800250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:28.772948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.809875+0000","last_change":"2024-04-25T13:17:39.816586+0000","last_active":"2024-04-25T13:17:42.809875+0000","last_peered":"2024-04-25T13:17:42.809875+0000","last_clean":"2024-04-25T13:17:42.809875+0000","last_became_active":"2024-04-25T13:17:39.815712+0000","last_became_peered":"2024-04-25T13:17:39.815712+0000","last_unstale":"2024-04-25T13:17:42.809875+0000","last_undegraded":"2024-04-25T13:17:42.809875+0000","last_fullsized":"2024-04-25T13:17:42.809875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.772789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817304+0000","last_change":"2024-04-25T13:17:40.824505+0000","last_active":"2024-04-25T13:17:41.817304+0000","last_peered":"2024-04-25T13:17:41.817304+0000","last_clean":"2024-04-25T13:17:41.817304+0000","last_became_active":"2024-04-25T13:17:40.824197+0000","last_became_peered":"2024-04-25T13:17:40.824197+0000","last_unstale":"2024-04-25T13:17:41.817304+0000","last_undegraded":"2024-04-25T13:17:41.817304+0000","last_fullsized":"2024-04-25T13:17:41.817304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:17.484269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.811988+0000","last_change":"2024-04-25T13:17:39.810242+0000","last_active":"2024-04-25T13:17:40.811988+0000","last_peered":"2024-04-25T13:17:40.811988+0000","last_clean":"2024-04-25T13:17:40.811988+0000","last_became_active":"2024-04-25T13:17:39.809547+0000","last_became_peered":"2024-04-25T13:17:39.809547+0000","last_unstale":"2024-04-25T13:17:40.811988+0000","last_undegraded":"2024-04-25T13:17:40.811988+0000","last_fullsized":"2024-04-25T13:17:40.811988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.615321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810386+0000","last_change":"2024-04-25T13:17:40.822114+0000","last_active":"2024-04-25T13:17:42.810386+0000","last_peered":"2024-04-25T13:17:42.810386+0000","last_clean":"2024-04-25T13:17:42.810386+0000","last_became_active":"2024-04-25T13:17:40.821517+0000","last_became_peered":"2024-04-25T13:17:40.821517+0000","last_unstale":"2024-04-25T13:17:42.810386+0000","last_undegraded":"2024-04-25T13:17:42.810386+0000","last_fullsized":"2024-04-25T13:17:42.810386+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:07.225119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800304+0000","last_change":"2024-04-25T13:17:39.812135+0000","last_active":"2024-04-25T13:17:42.800304+0000","last_peered":"2024-04-25T13:17:42.800304+0000","last_clean":"2024-04-25T13:17:42.800304+0000","last_became_active":"2024-04-25T13:17:39.811836+0000","last_became_peered":"2024-04-25T13:17:39.811836+0000","last_unstale":"2024-04-25T13:17:42.800304+0000","last_undegraded":"2024-04-25T13:17:42.800304+0000","last_fullsized":"2024-04-25T13:17:42.800304+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:00.692725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283788+0000","last_change":"2024-04-25T13:17:40.830160+0000","last_active":"2024-04-25T13:17:42.283788+0000","last_peered":"2024-04-25T13:17:42.283788+0000","last_clean":"2024-04-25T13:17:42.283788+0000","last_became_active":"2024-04-25T13:17:40.829773+0000","last_became_peered":"2024-04-25T13:17:40.829773+0000","last_unstale":"2024-04-25T13:17:42.283788+0000","last_undegraded":"2024-04-25T13:17:42.283788+0000","last_fullsized":"2024-04-25T13:17:42.283788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:24.515333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.810226+0000","last_change":"2024-04-25T13:17:39.819011+0000","last_active":"2024-04-25T13:17:40.810226+0000","last_peered":"2024-04-25T13:17:40.810226+0000","last_clean":"2024-04-25T13:17:40.810226+0000","last_became_active":"2024-04-25T13:17:39.818656+0000","last_became_peered":"2024-04-25T13:17:39.818656+0000","last_unstale":"2024-04-25T13:17:40.810226+0000","last_undegraded":"2024-04-25T13:17:40.810226+0000","last_fullsized":"2024-04-25T13:17:40.810226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:20.507500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.284048+0000","last_change":"2024-04-25T13:17:40.829595+0000","last_active":"2024-04-25T13:17:42.284048+0000","last_peered":"2024-04-25T13:17:42.284048+0000","last_clean":"2024-04-25T13:17:42.284048+0000","last_became_active":"2024-04-25T13:17:40.827707+0000","last_became_peered":"2024-04-25T13:17:40.827707+0000","last_unstale":"2024-04-25T13:17:42.284048+0000","last_undegraded":"2024-04-25T13:17:42.284048+0000","last_fullsized":"2024-04-25T13:17:42.284048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:49.002629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.829658+0000","last_change":"2024-04-25T13:17:39.809740+0000","last_active":"2024-04-25T13:17:40.829658+0000","last_peered":"2024-04-25T13:17:40.829658+0000","last_clean":"2024-04-25T13:17:40.829658+0000","last_became_active":"2024-04-25T13:17:39.809407+0000","last_became_peered":"2024-04-25T13:17:39.809407+0000","last_unstale":"2024-04-25T13:17:40.829658+0000","last_undegraded":"2024-04-25T13:17:40.829658+0000","last_fullsized":"2024-04-25T13:17:40.829658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:16.732755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283716+0000","last_change":"2024-04-25T13:17:40.825573+0000","last_active":"2024-04-25T13:17:42.283716+0000","last_peered":"2024-04-25T13:17:42.283716+0000","last_clean":"2024-04-25T13:17:42.283716+0000","last_became_active":"2024-04-25T13:17:40.825181+0000","last_became_peered":"2024-04-25T13:17:40.825181+0000","last_unstale":"2024-04-25T13:17:42.283716+0000","last_undegraded":"2024-04-25T13:17:42.283716+0000","last_fullsized":"2024-04-25T13:17:42.283716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:06.446494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311533+0000","last_change":"2024-04-25T13:17:39.812458+0000","last_active":"2024-04-25T13:17:43.311533+0000","last_peered":"2024-04-25T13:17:43.311533+0000","last_clean":"2024-04-25T13:17:43.311533+0000","last_became_active":"2024-04-25T13:17:39.812092+0000","last_became_peered":"2024-04-25T13:17:39.812092+0000","last_unstale":"2024-04-25T13:17:43.311533+0000","last_undegraded":"2024-04-25T13:17:43.311533+0000","last_fullsized":"2024-04-25T13:17:43.311533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:07.210444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.284039+0000","last_change":"2024-04-25T13:17:40.821510+0000","last_active":"2024-04-25T13:17:42.284039+0000","last_peered":"2024-04-25T13:17:42.284039+0000","last_clean":"2024-04-25T13:17:42.284039+0000","last_became_active":"2024-04-25T13:17:40.818897+0000","last_became_peered":"2024-04-25T13:17:40.818897+0000","last_unstale":"2024-04-25T13:17:42.284039+0000","last_undegraded":"2024-04-25T13:17:42.284039+0000","last_fullsized":"2024-04-25T13:17:42.284039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:15.215991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817436+0000","last_change":"2024-04-25T13:17:40.823789+0000","last_active":"2024-04-25T13:17:41.817436+0000","last_peered":"2024-04-25T13:17:41.817436+0000","last_clean":"2024-04-25T13:17:41.817436+0000","last_became_active":"2024-04-25T13:17:40.823424+0000","last_became_peered":"2024-04-25T13:17:40.823424+0000","last_unstale":"2024-04-25T13:17:41.817436+0000","last_undegraded":"2024-04-25T13:17:41.817436+0000","last_fullsized":"2024-04-25T13:17:41.817436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:50.022286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.827576+0000","last_change":"2024-04-25T13:17:40.827085+0000","last_active":"2024-04-25T13:17:40.827576+0000","last_peered":"2024-04-25T13:17:40.827576+0000","last_clean":"2024-04-25T13:17:40.827576+0000","last_became_active":"2024-04-25T13:17:40.826760+0000","last_became_peered":"2024-04-25T13:17:40.826760+0000","last_unstale":"2024-04-25T13:17:40.827576+0000","last_undegraded":"2024-04-25T13:17:40.827576+0000","last_fullsized":"2024-04-25T13:17:40.827576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:51.133281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.825244+0000","last_change":"2024-04-25T13:17:40.824283+0000","last_active":"2024-04-25T13:17:40.825244+0000","last_peered":"2024-04-25T13:17:40.825244+0000","last_clean":"2024-04-25T13:17:40.825244+0000","last_became_active":"2024-04-25T13:17:40.823914+0000","last_became_peered":"2024-04-25T13:17:40.823914+0000","last_unstale":"2024-04-25T13:17:40.825244+0000","last_undegraded":"2024-04-25T13:17:40.825244+0000","last_fullsized":"2024-04-25T13:17:40.825244+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:40.113901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.823265+0000","last_change":"2024-04-25T13:17:40.822003+0000","last_active":"2024-04-25T13:17:40.823265+0000","last_peered":"2024-04-25T13:17:40.823265+0000","last_clean":"2024-04-25T13:17:40.823265+0000","last_became_active":"2024-04-25T13:17:40.821453+0000","last_became_peered":"2024-04-25T13:17:40.821453+0000","last_unstale":"2024-04-25T13:17:40.823265+0000","last_undegraded":"2024-04-25T13:17:40.823265+0000","last_fullsized":"2024-04-25T13:17:40.823265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:06.226983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311730+0000","last_change":"2024-04-25T13:17:40.832449+0000","last_active":"2024-04-25T13:17:43.311730+0000","last_peered":"2024-04-25T13:17:43.311730+0000","last_clean":"2024-04-25T13:17:43.311730+0000","last_became_active":"2024-04-25T13:17:40.832079+0000","last_became_peered":"2024-04-25T13:17:40.832079+0000","last_unstale":"2024-04-25T13:17:43.311730+0000","last_undegraded":"2024-04-25T13:17:43.311730+0000","last_fullsized":"2024-04-25T13:17:43.311730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:44.537129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817569+0000","last_change":"2024-04-25T13:17:40.828466+0000","last_active":"2024-04-25T13:17:41.817569+0000","last_peered":"2024-04-25T13:17:41.817569+0000","last_clean":"2024-04-25T13:17:41.817569+0000","last_became_active":"2024-04-25T13:17:40.827996+0000","last_became_peered":"2024-04-25T13:17:40.827996+0000","last_unstale":"2024-04-25T13:17:41.817569+0000","last_undegraded":"2024-04-25T13:17:41.817569+0000","last_fullsized":"2024-04-25T13:17:41.817569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:26.498795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.823122+0000","last_change":"2024-04-25T13:17:40.821024+0000","last_active":"2024-04-25T13:17:40.823122+0000","last_peered":"2024-04-25T13:17:40.823122+0000","last_clean":"2024-04-25T13:17:40.823122+0000","last_became_active":"2024-04-25T13:17:40.820671+0000","last_became_peered":"2024-04-25T13:17:40.820671+0000","last_unstale":"2024-04-25T13:17:40.823122+0000","last_undegraded":"2024-04-25T13:17:40.823122+0000","last_fullsized":"2024-04-25T13:17:40.823122+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:04.426073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810613+0000","last_change":"2024-04-25T13:17:40.837131+0000","last_active":"2024-04-25T13:17:42.810613+0000","last_peered":"2024-04-25T13:17:42.810613+0000","last_clean":"2024-04-25T13:17:42.810613+0000","last_became_active":"2024-04-25T13:17:40.836784+0000","last_became_peered":"2024-04-25T13:17:40.836784+0000","last_unstale":"2024-04-25T13:17:42.810613+0000","last_undegraded":"2024-04-25T13:17:42.810613+0000","last_fullsized":"2024-04-25T13:17:42.810613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:41.818001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.828923+0000","last_change":"2024-04-25T13:17:40.827149+0000","last_active":"2024-04-25T13:17:40.828923+0000","last_peered":"2024-04-25T13:17:40.828923+0000","last_clean":"2024-04-25T13:17:40.828923+0000","last_became_active":"2024-04-25T13:17:40.826800+0000","last_became_peered":"2024-04-25T13:17:40.826800+0000","last_unstale":"2024-04-25T13:17:40.828923+0000","last_undegraded":"2024-04-25T13:17:40.828923+0000","last_fullsized":"2024-04-25T13:17:40.828923+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:01.165866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.809855+0000","last_change":"2024-04-25T13:17:40.825028+0000","last_active":"2024-04-25T13:17:42.809855+0000","last_peered":"2024-04-25T13:17:42.809855+0000","last_clean":"2024-04-25T13:17:42.809855+0000","last_became_active":"2024-04-25T13:17:40.824614+0000","last_became_peered":"2024-04-25T13:17:40.824614+0000","last_unstale":"2024-04-25T13:17:42.809855+0000","last_undegraded":"2024-04-25T13:17:42.809855+0000","last_fullsized":"2024-04-25T13:17:42.809855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:07.629080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T13:17:40.836824+0000","last_change":"2024-04-25T13:17:40.836345+0000","last_active":"2024-04-25T13:17:40.836824+0000","last_peered":"2024-04-25T13:17:40.836824+0000","last_clean":"2024-04-25T13:17:40.836824+0000","last_became_active":"2024-04-25T13:17:40.835989+0000","last_became_peered":"2024-04-25T13:17:40.835989+0000","last_unstale":"2024-04-25T13:17:40.836824+0000","last_undegraded":"2024-04-25T13:17:40.836824+0000","last_fullsized":"2024-04-25T13:17:40.836824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:33.462484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311818+0000","last_change":"2024-04-25T13:17:40.835190+0000","last_active":"2024-04-25T13:17:43.311818+0000","last_peered":"2024-04-25T13:17:43.311818+0000","last_clean":"2024-04-25T13:17:43.311818+0000","last_became_active":"2024-04-25T13:17:40.834847+0000","last_became_peered":"2024-04-25T13:17:40.834847+0000","last_unstale":"2024-04-25T13:17:43.311818+0000","last_undegraded":"2024-04-25T13:17:43.311818+0000","last_fullsized":"2024-04-25T13:17:43.311818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:18.217464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817389+0000","last_change":"2024-04-25T13:17:40.827578+0000","last_active":"2024-04-25T13:17:41.817389+0000","last_peered":"2024-04-25T13:17:41.817389+0000","last_clean":"2024-04-25T13:17:41.817389+0000","last_became_active":"2024-04-25T13:17:40.827256+0000","last_became_peered":"2024-04-25T13:17:40.827256+0000","last_unstale":"2024-04-25T13:17:41.817389+0000","last_undegraded":"2024-04-25T13:17:41.817389+0000","last_fullsized":"2024-04-25T13:17:41.817389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:40.738629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":7230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":451857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547217,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36868,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708284,"statfs":{"total":95995035648,"available":95957282816,"internally_reserved":0,"allocated":1560576,"data_stored":1044031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743448,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":452439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":950272,"data_stored":452461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266721,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1540096,"data_stored":1042807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":443287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":444899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":434210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":434210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117047,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":950272,"data_stored":451961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":933888,"data_stored":443309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691884,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36840,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708312,"statfs":{"total":95995035648,"available":95957311488,"internally_reserved":0,"allocated":1536000,"data_stored":1033837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:17:47.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.788+0000 7fef7c929640 1 -- 172.21.15.31:0/871713510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef78100e90 msgr2=0x7fef781012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.788+0000 7fef7c929640 1 --2- 172.21.15.31:0/871713510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef78100e90 0x7fef781012b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009900 tx=0x7fef6c02a410 comp rx=0 tx=0).stop 2024-04-25T13:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.788+0000 7fef7c929640 1 -- 172.21.15.31:0/871713510 shutdown_connections 2024-04-25T13:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.788+0000 7fef7c929640 1 --2- 172.21.15.31:0/871713510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef781029a0 0x7fef781092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.788+0000 7fef7c929640 1 --2- 172.21.15.31:0/871713510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef78101fe0 0x7fef78102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.788+0000 7fef7c929640 1 --2- 172.21.15.31:0/871713510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef78100e90 0x7fef781012b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.788+0000 7fef7c929640 1 -- 172.21.15.31:0/871713510 >> 172.21.15.31:0/871713510 conn(0x7fef780fc400 msgr2=0x7fef780fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.789+0000 7fef7c929640 1 -- 172.21.15.31:0/871713510 shutdown_connections 2024-04-25T13:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.789+0000 7fef7c929640 1 -- 172.21.15.31:0/871713510 wait complete. 2024-04-25T13:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.789+0000 7fef7c929640 1 Processor -- start 2024-04-25T13:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.789+0000 7fef7c929640 1 -- start start 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.789+0000 7fef7c929640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef78100e90 0x7fef7819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.789+0000 7fef7c929640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef78101fe0 0x7fef7819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.789+0000 7fef7c929640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef781029a0 0x7fef781a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.789+0000 7fef7c929640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fef7810c9e0 con 0x7fef78101fe0 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.789+0000 7fef7c929640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fef7810c860 con 0x7fef781029a0 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.789+0000 7fef7c929640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef7810cb60 con 0x7fef78100e90 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.789+0000 7fef75d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef78101fe0 0x7fef7819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef75d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef78101fe0 0x7fef7819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50918/0 (socket says 172.21.15.31:50918) 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef75d9b640 1 -- 172.21.15.31:0/2210319233 learned_addr learned my addr 172.21.15.31:0/2210319233 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef7659c640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef78100e90 0x7fef7819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef75d9b640 1 -- 172.21.15.31:0/2210319233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef78100e90 msgr2=0x7fef7819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef76d9d640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef781029a0 0x7fef781a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef75d9b640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef78100e90 0x7fef7819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef75d9b640 1 -- 172.21.15.31:0/2210319233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef781029a0 msgr2=0x7fef781a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef75d9b640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef781029a0 0x7fef781a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef75d9b640 1 -- 172.21.15.31:0/2210319233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef781a2040 con 0x7fef78101fe0 2024-04-25T13:17:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef7659c640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef78100e90 0x7fef7819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef75d9b640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef78101fe0 0x7fef7819a960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fef6800d950 tx=0x7fef6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef677fe640 1 -- 172.21.15.31:0/2210319233 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68017070 con 0x7fef78101fe0 2024-04-25T13:17:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef677fe640 1 -- 172.21.15.31:0/2210319233 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef68009c30 con 0x7fef78101fe0 2024-04-25T13:17:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef7c929640 1 -- 172.21.15.31:0/2210319233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef781a2330 con 0x7fef78101fe0 2024-04-25T13:17:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef677fe640 1 -- 172.21.15.31:0/2210319233 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6800f620 con 0x7fef78101fe0 2024-04-25T13:17:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.790+0000 7fef7c929640 1 -- 172.21.15.31:0/2210319233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef781a2830 con 0x7fef78101fe0 2024-04-25T13:17:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.791+0000 7fef7c929640 1 -- 172.21.15.31:0/2210319233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef781a2a90 con 0x7fef78101fe0 2024-04-25T13:17:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.792+0000 7fef677fe640 1 -- 172.21.15.31:0/2210319233 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fef6801a070 con 0x7fef78101fe0 2024-04-25T13:17:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.793+0000 7fef677fe640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fef600a3c40 0x7fef600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.793+0000 7fef7659c640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fef600a3c40 0x7fef600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.793+0000 7fef677fe640 1 -- 172.21.15.31:0/2210319233 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fef680cfed0 con 0x7fef78101fe0 2024-04-25T13:17:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.793+0000 7fef7659c640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fef600a3c40 0x7fef600a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006010 tx=0x7fef6c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:47.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.797+0000 7fef677fe640 1 -- 172.21.15.31:0/2210319233 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef68009de0 con 0x7fef78101fe0 2024-04-25T13:17:47.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:47 smithi031 ceph-mon[29457]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:47.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:47 smithi031 ceph-mon[29457]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:17:47.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:47 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/910220169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:47.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:47 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2127621898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:47.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.985+0000 7fef7c929640 1 -- 172.21.15.31:0/2210319233 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef780698e0 con 0x7fef600a3c40 2024-04-25T13:17:47.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.992+0000 7fef677fe640 1 -- 172.21.15.31:0/2210319233 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258681 (secure 0 0 0) 0x7fef780698e0 con 0x7fef600a3c40 2024-04-25T13:17:47.995 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:47.996 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T13:17:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.995+0000 7fef7c929640 1 -- 172.21.15.31:0/2210319233 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fef600a3c40 msgr2=0x7fef600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.995+0000 7fef7c929640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fef600a3c40 0x7fef600a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006010 tx=0x7fef6c000bc0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.995+0000 7fef7c929640 1 -- 172.21.15.31:0/2210319233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef78101fe0 msgr2=0x7fef7819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.995+0000 7fef7c929640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef78101fe0 0x7fef7819a960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fef6800d950 tx=0x7fef6800de10 comp rx=0 tx=0).stop 2024-04-25T13:17:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.995+0000 7fef7c929640 1 -- 172.21.15.31:0/2210319233 shutdown_connections 2024-04-25T13:17:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.995+0000 7fef7c929640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fef600a3c40 0x7fef600a6100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.995+0000 7fef7c929640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef781029a0 0x7fef781a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.995+0000 7fef7c929640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef78101fe0 0x7fef7819a960 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.996+0000 7fef7c929640 1 --2- 172.21.15.31:0/2210319233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef78100e90 0x7fef7819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.996+0000 7fef7c929640 1 -- 172.21.15.31:0/2210319233 >> 172.21.15.31:0/2210319233 conn(0x7fef780fc400 msgr2=0x7fef780691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.996+0000 7fef7c929640 1 -- 172.21.15.31:0/2210319233 shutdown_connections 2024-04-25T13:17:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:47.996+0000 7fef7c929640 1 -- 172.21.15.31:0/2210319233 wait complete. 2024-04-25T13:17:48.007 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-04-25T13:17:47.802087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":142,"num_read_kb":125,"num_write":226,"num_write_kb":2952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176,"ondisk_log_size":176,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382892,"kb_used_data":12888,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558932,"statfs":{"total":1151940427776,"available":1151548346368,"internally_reserved":0,"allocated":13197312,"data_stored":7222161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001868"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817724+0000","last_change":"2024-04-25T13:17:40.825728+0000","last_active":"2024-04-25T13:17:41.817724+0000","last_peered":"2024-04-25T13:17:41.817724+0000","last_clean":"2024-04-25T13:17:41.817724+0000","last_became_active":"2024-04-25T13:17:40.824933+0000","last_became_peered":"2024-04-25T13:17:40.824933+0000","last_unstale":"2024-04-25T13:17:41.817724+0000","last_undegraded":"2024-04-25T13:17:41.817724+0000","last_fullsized":"2024-04-25T13:17:41.817724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:14.326614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.626521+0000","last_change":"2024-04-25T13:17:40.821148+0000","last_active":"2024-04-25T13:17:42.626521+0000","last_peered":"2024-04-25T13:17:42.626521+0000","last_clean":"2024-04-25T13:17:42.626521+0000","last_became_active":"2024-04-25T13:17:40.820244+0000","last_became_peered":"2024-04-25T13:17:40.820244+0000","last_unstale":"2024-04-25T13:17:42.626521+0000","last_undegraded":"2024-04-25T13:17:42.626521+0000","last_fullsized":"2024-04-25T13:17:42.626521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.566793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.626333+0000","last_change":"2024-04-25T13:17:40.818729+0000","last_active":"2024-04-25T13:17:42.626333+0000","last_peered":"2024-04-25T13:17:42.626333+0000","last_clean":"2024-04-25T13:17:42.626333+0000","last_became_active":"2024-04-25T13:17:40.818351+0000","last_became_peered":"2024-04-25T13:17:40.818351+0000","last_unstale":"2024-04-25T13:17:42.626333+0000","last_undegraded":"2024-04-25T13:17:42.626333+0000","last_fullsized":"2024-04-25T13:17:42.626333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:42.412891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.543414+0000","last_change":"2024-04-25T13:17:40.820932+0000","last_active":"2024-04-25T13:17:43.543414+0000","last_peered":"2024-04-25T13:17:43.543414+0000","last_clean":"2024-04-25T13:17:43.543414+0000","last_became_active":"2024-04-25T13:17:40.820525+0000","last_became_peered":"2024-04-25T13:17:40.820525+0000","last_unstale":"2024-04-25T13:17:43.543414+0000","last_undegraded":"2024-04-25T13:17:43.543414+0000","last_fullsized":"2024-04-25T13:17:43.543414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:12.077703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800540+0000","last_change":"2024-04-25T13:17:40.829959+0000","last_active":"2024-04-25T13:17:42.800540+0000","last_peered":"2024-04-25T13:17:42.800540+0000","last_clean":"2024-04-25T13:17:42.800540+0000","last_became_active":"2024-04-25T13:17:40.829594+0000","last_became_peered":"2024-04-25T13:17:40.829594+0000","last_unstale":"2024-04-25T13:17:42.800540+0000","last_undegraded":"2024-04-25T13:17:42.800540+0000","last_fullsized":"2024-04-25T13:17:42.800540+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:55.719296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821448+0000","last_change":"2024-04-25T13:17:40.817436+0000","last_active":"2024-04-25T13:17:41.821448+0000","last_peered":"2024-04-25T13:17:41.821448+0000","last_clean":"2024-04-25T13:17:41.821448+0000","last_became_active":"2024-04-25T13:17:40.816763+0000","last_became_peered":"2024-04-25T13:17:40.816763+0000","last_unstale":"2024-04-25T13:17:41.821448+0000","last_undegraded":"2024-04-25T13:17:41.821448+0000","last_fullsized":"2024-04-25T13:17:41.821448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:09.505890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283501+0000","last_change":"2024-04-25T13:17:40.822968+0000","last_active":"2024-04-25T13:17:42.283501+0000","last_peered":"2024-04-25T13:17:42.283501+0000","last_clean":"2024-04-25T13:17:42.283501+0000","last_became_active":"2024-04-25T13:17:40.822600+0000","last_became_peered":"2024-04-25T13:17:40.822600+0000","last_unstale":"2024-04-25T13:17:42.283501+0000","last_undegraded":"2024-04-25T13:17:42.283501+0000","last_fullsized":"2024-04-25T13:17:42.283501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:08:32.276310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.809857+0000","last_change":"2024-04-25T13:17:40.823421+0000","last_active":"2024-04-25T13:17:42.809857+0000","last_peered":"2024-04-25T13:17:42.809857+0000","last_clean":"2024-04-25T13:17:42.809857+0000","last_became_active":"2024-04-25T13:17:40.823054+0000","last_became_peered":"2024-04-25T13:17:40.823054+0000","last_unstale":"2024-04-25T13:17:42.809857+0000","last_undegraded":"2024-04-25T13:17:42.809857+0000","last_fullsized":"2024-04-25T13:17:42.809857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:34.256763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.543523+0000","last_change":"2024-04-25T13:17:40.826105+0000","last_active":"2024-04-25T13:17:43.543523+0000","last_peered":"2024-04-25T13:17:43.543523+0000","last_clean":"2024-04-25T13:17:43.543523+0000","last_became_active":"2024-04-25T13:17:40.825763+0000","last_became_peered":"2024-04-25T13:17:40.825763+0000","last_unstale":"2024-04-25T13:17:43.543523+0000","last_undegraded":"2024-04-25T13:17:43.543523+0000","last_fullsized":"2024-04-25T13:17:43.543523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.058115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821749+0000","last_change":"2024-04-25T13:17:40.817356+0000","last_active":"2024-04-25T13:17:41.821749+0000","last_peered":"2024-04-25T13:17:41.821749+0000","last_clean":"2024-04-25T13:17:41.821749+0000","last_became_active":"2024-04-25T13:17:40.816906+0000","last_became_peered":"2024-04-25T13:17:40.816906+0000","last_unstale":"2024-04-25T13:17:41.821749+0000","last_undegraded":"2024-04-25T13:17:41.821749+0000","last_fullsized":"2024-04-25T13:17:41.821749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283927+0000","last_change":"2024-04-25T13:17:40.829445+0000","last_active":"2024-04-25T13:17:42.283927+0000","last_peered":"2024-04-25T13:17:42.283927+0000","last_clean":"2024-04-25T13:17:42.283927+0000","last_became_active":"2024-04-25T13:17:40.826709+0000","last_became_peered":"2024-04-25T13:17:40.826709+0000","last_unstale":"2024-04-25T13:17:42.283927+0000","last_undegraded":"2024-04-25T13:17:42.283927+0000","last_fullsized":"2024-04-25T13:17:42.283927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:43.044192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.543222+0000","last_change":"2024-04-25T13:17:40.827486+0000","last_active":"2024-04-25T13:17:43.543222+0000","last_peered":"2024-04-25T13:17:43.543222+0000","last_clean":"2024-04-25T13:17:43.543222+0000","last_became_active":"2024-04-25T13:17:40.827070+0000","last_became_peered":"2024-04-25T13:17:40.827070+0000","last_unstale":"2024-04-25T13:17:43.543222+0000","last_undegraded":"2024-04-25T13:17:43.543222+0000","last_fullsized":"2024-04-25T13:17:43.543222+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:39.564567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311636+0000","last_change":"2024-04-25T13:17:40.826645+0000","last_active":"2024-04-25T13:17:43.311636+0000","last_peered":"2024-04-25T13:17:43.311636+0000","last_clean":"2024-04-25T13:17:43.311636+0000","last_became_active":"2024-04-25T13:17:40.826314+0000","last_became_peered":"2024-04-25T13:17:40.826314+0000","last_unstale":"2024-04-25T13:17:43.311636+0000","last_undegraded":"2024-04-25T13:17:43.311636+0000","last_fullsized":"2024-04-25T13:17:43.311636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:59.600451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.626109+0000","last_change":"2024-04-25T13:17:40.814918+0000","last_active":"2024-04-25T13:17:42.626109+0000","last_peered":"2024-04-25T13:17:42.626109+0000","last_clean":"2024-04-25T13:17:42.626109+0000","last_became_active":"2024-04-25T13:17:40.814573+0000","last_became_peered":"2024-04-25T13:17:40.814573+0000","last_unstale":"2024-04-25T13:17:42.626109+0000","last_undegraded":"2024-04-25T13:17:42.626109+0000","last_fullsized":"2024-04-25T13:17:42.626109+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:34.554390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.626024+0000","last_change":"2024-04-25T13:17:40.825481+0000","last_active":"2024-04-25T13:17:42.626024+0000","last_peered":"2024-04-25T13:17:42.626024+0000","last_clean":"2024-04-25T13:17:42.626024+0000","last_became_active":"2024-04-25T13:17:40.823726+0000","last_became_peered":"2024-04-25T13:17:40.823726+0000","last_unstale":"2024-04-25T13:17:42.626024+0000","last_undegraded":"2024-04-25T13:17:42.626024+0000","last_fullsized":"2024-04-25T13:17:42.626024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:46.500031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.542541+0000","last_change":"2024-04-25T13:17:40.827561+0000","last_active":"2024-04-25T13:17:43.542541+0000","last_peered":"2024-04-25T13:17:43.542541+0000","last_clean":"2024-04-25T13:17:43.542541+0000","last_became_active":"2024-04-25T13:17:40.827100+0000","last_became_peered":"2024-04-25T13:17:40.827100+0000","last_unstale":"2024-04-25T13:17:43.542541+0000","last_undegraded":"2024-04-25T13:17:43.542541+0000","last_fullsized":"2024-04-25T13:17:43.542541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:41.381469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817824+0000","last_change":"2024-04-25T13:17:40.826017+0000","last_active":"2024-04-25T13:17:41.817824+0000","last_peered":"2024-04-25T13:17:41.817824+0000","last_clean":"2024-04-25T13:17:41.817824+0000","last_became_active":"2024-04-25T13:17:40.825429+0000","last_became_peered":"2024-04-25T13:17:40.825429+0000","last_unstale":"2024-04-25T13:17:41.817824+0000","last_undegraded":"2024-04-25T13:17:41.817824+0000","last_fullsized":"2024-04-25T13:17:41.817824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:58.788260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.816491+0000","last_change":"2024-04-25T13:17:40.830115+0000","last_active":"2024-04-25T13:17:41.816491+0000","last_peered":"2024-04-25T13:17:41.816491+0000","last_clean":"2024-04-25T13:17:41.816491+0000","last_became_active":"2024-04-25T13:17:40.829820+0000","last_became_peered":"2024-04-25T13:17:40.829820+0000","last_unstale":"2024-04-25T13:17:41.816491+0000","last_undegraded":"2024-04-25T13:17:41.816491+0000","last_fullsized":"2024-04-25T13:17:41.816491+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:35.051183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.851006+0000","last_change":"2024-04-25T13:17:40.824689+0000","last_active":"2024-04-25T13:17:41.851006+0000","last_peered":"2024-04-25T13:17:41.851006+0000","last_clean":"2024-04-25T13:17:41.851006+0000","last_became_active":"2024-04-25T13:17:40.823878+0000","last_became_peered":"2024-04-25T13:17:40.823878+0000","last_unstale":"2024-04-25T13:17:41.851006+0000","last_undegraded":"2024-04-25T13:17:41.851006+0000","last_fullsized":"2024-04-25T13:17:41.851006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:23.745510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.816449+0000","last_change":"2024-04-25T13:17:39.812816+0000","last_active":"2024-04-25T13:17:41.816449+0000","last_peered":"2024-04-25T13:17:41.816449+0000","last_clean":"2024-04-25T13:17:41.816449+0000","last_became_active":"2024-04-25T13:17:39.812483+0000","last_became_peered":"2024-04-25T13:17:39.812483+0000","last_unstale":"2024-04-25T13:17:41.816449+0000","last_undegraded":"2024-04-25T13:17:41.816449+0000","last_fullsized":"2024-04-25T13:17:41.816449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:14.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810295+0000","last_change":"2024-04-25T13:17:40.824764+0000","last_active":"2024-04-25T13:17:42.810295+0000","last_peered":"2024-04-25T13:17:42.810295+0000","last_clean":"2024-04-25T13:17:42.810295+0000","last_became_active":"2024-04-25T13:17:40.824337+0000","last_became_peered":"2024-04-25T13:17:40.824337+0000","last_unstale":"2024-04-25T13:17:42.810295+0000","last_undegraded":"2024-04-25T13:17:42.810295+0000","last_fullsized":"2024-04-25T13:17:42.810295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:37.125246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810544+0000","last_change":"2024-04-25T13:17:39.819820+0000","last_active":"2024-04-25T13:17:42.810544+0000","last_peered":"2024-04-25T13:17:42.810544+0000","last_clean":"2024-04-25T13:17:42.810544+0000","last_became_active":"2024-04-25T13:17:39.819446+0000","last_became_peered":"2024-04-25T13:17:39.819446+0000","last_unstale":"2024-04-25T13:17:42.810544+0000","last_undegraded":"2024-04-25T13:17:42.810544+0000","last_fullsized":"2024-04-25T13:17:42.810544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:36.902011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.626720+0000","last_change":"2024-04-25T13:17:40.821183+0000","last_active":"2024-04-25T13:17:42.626720+0000","last_peered":"2024-04-25T13:17:42.626720+0000","last_clean":"2024-04-25T13:17:42.626720+0000","last_became_active":"2024-04-25T13:17:40.820789+0000","last_became_peered":"2024-04-25T13:17:40.820789+0000","last_unstale":"2024-04-25T13:17:42.626720+0000","last_undegraded":"2024-04-25T13:17:42.626720+0000","last_fullsized":"2024-04-25T13:17:42.626720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:56.225385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800656+0000","last_change":"2024-04-25T13:17:39.813132+0000","last_active":"2024-04-25T13:17:42.800656+0000","last_peered":"2024-04-25T13:17:42.800656+0000","last_clean":"2024-04-25T13:17:42.800656+0000","last_became_active":"2024-04-25T13:17:39.812837+0000","last_became_peered":"2024-04-25T13:17:39.812837+0000","last_unstale":"2024-04-25T13:17:42.800656+0000","last_undegraded":"2024-04-25T13:17:42.800656+0000","last_fullsized":"2024-04-25T13:17:42.800656+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:55.342376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.542867+0000","last_change":"2024-04-25T13:17:40.824143+0000","last_active":"2024-04-25T13:17:43.542867+0000","last_peered":"2024-04-25T13:17:43.542867+0000","last_clean":"2024-04-25T13:17:43.542867+0000","last_became_active":"2024-04-25T13:17:40.823778+0000","last_became_peered":"2024-04-25T13:17:40.823778+0000","last_unstale":"2024-04-25T13:17:43.542867+0000","last_undegraded":"2024-04-25T13:17:43.542867+0000","last_fullsized":"2024-04-25T13:17:43.542867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:54.112957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800644+0000","last_change":"2024-04-25T13:17:39.817743+0000","last_active":"2024-04-25T13:17:42.800644+0000","last_peered":"2024-04-25T13:17:42.800644+0000","last_clean":"2024-04-25T13:17:42.800644+0000","last_became_active":"2024-04-25T13:17:39.817392+0000","last_became_peered":"2024-04-25T13:17:39.817392+0000","last_unstale":"2024-04-25T13:17:42.800644+0000","last_undegraded":"2024-04-25T13:17:42.800644+0000","last_fullsized":"2024-04-25T13:17:42.800644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.400644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.966296+0000","last_change":"2024-04-25T13:17:40.829895+0000","last_active":"2024-04-25T13:17:42.966296+0000","last_peered":"2024-04-25T13:17:42.966296+0000","last_clean":"2024-04-25T13:17:42.966296+0000","last_became_active":"2024-04-25T13:17:40.829300+0000","last_became_peered":"2024-04-25T13:17:40.829300+0000","last_unstale":"2024-04-25T13:17:42.966296+0000","last_undegraded":"2024-04-25T13:17:42.966296+0000","last_fullsized":"2024-04-25T13:17:42.966296+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:49.935081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.966281+0000","last_change":"2024-04-25T13:17:39.813231+0000","last_active":"2024-04-25T13:17:42.966281+0000","last_peered":"2024-04-25T13:17:42.966281+0000","last_clean":"2024-04-25T13:17:42.966281+0000","last_became_active":"2024-04-25T13:17:39.812441+0000","last_became_peered":"2024-04-25T13:17:39.812441+0000","last_unstale":"2024-04-25T13:17:42.966281+0000","last_undegraded":"2024-04-25T13:17:42.966281+0000","last_fullsized":"2024-04-25T13:17:42.966281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:17.578986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800414+0000","last_change":"2024-04-25T13:17:40.823309+0000","last_active":"2024-04-25T13:17:42.800414+0000","last_peered":"2024-04-25T13:17:42.800414+0000","last_clean":"2024-04-25T13:17:42.800414+0000","last_became_active":"2024-04-25T13:17:40.822909+0000","last_became_peered":"2024-04-25T13:17:40.822909+0000","last_unstale":"2024-04-25T13:17:42.800414+0000","last_undegraded":"2024-04-25T13:17:42.800414+0000","last_fullsized":"2024-04-25T13:17:42.800414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:07.124306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821197+0000","last_change":"2024-04-25T13:17:39.809888+0000","last_active":"2024-04-25T13:17:41.821197+0000","last_peered":"2024-04-25T13:17:41.821197+0000","last_clean":"2024-04-25T13:17:41.821197+0000","last_became_active":"2024-04-25T13:17:39.809519+0000","last_became_peered":"2024-04-25T13:17:39.809519+0000","last_unstale":"2024-04-25T13:17:41.821197+0000","last_undegraded":"2024-04-25T13:17:41.821197+0000","last_fullsized":"2024-04-25T13:17:41.821197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:43.306668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.966407+0000","last_change":"2024-04-25T13:17:40.828666+0000","last_active":"2024-04-25T13:17:42.966407+0000","last_peered":"2024-04-25T13:17:42.966407+0000","last_clean":"2024-04-25T13:17:42.966407+0000","last_became_active":"2024-04-25T13:17:40.828332+0000","last_became_peered":"2024-04-25T13:17:40.828332+0000","last_unstale":"2024-04-25T13:17:42.966407+0000","last_undegraded":"2024-04-25T13:17:42.966407+0000","last_fullsized":"2024-04-25T13:17:42.966407+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:03.089906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311412+0000","last_change":"2024-04-25T13:17:39.811231+0000","last_active":"2024-04-25T13:17:43.311412+0000","last_peered":"2024-04-25T13:17:43.311412+0000","last_clean":"2024-04-25T13:17:43.311412+0000","last_became_active":"2024-04-25T13:17:39.810762+0000","last_became_peered":"2024-04-25T13:17:39.810762+0000","last_unstale":"2024-04-25T13:17:43.311412+0000","last_undegraded":"2024-04-25T13:17:43.311412+0000","last_fullsized":"2024-04-25T13:17:43.311412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:15.970042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810115+0000","last_change":"2024-04-25T13:17:40.828074+0000","last_active":"2024-04-25T13:17:42.810115+0000","last_peered":"2024-04-25T13:17:42.810115+0000","last_clean":"2024-04-25T13:17:42.810115+0000","last_became_active":"2024-04-25T13:17:40.827702+0000","last_became_peered":"2024-04-25T13:17:40.827702+0000","last_unstale":"2024-04-25T13:17:42.810115+0000","last_undegraded":"2024-04-25T13:17:42.810115+0000","last_fullsized":"2024-04-25T13:17:42.810115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:23.408908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.965750+0000","last_change":"2024-04-25T13:17:39.812746+0000","last_active":"2024-04-25T13:17:42.965750+0000","last_peered":"2024-04-25T13:17:42.965750+0000","last_clean":"2024-04-25T13:17:42.965750+0000","last_became_active":"2024-04-25T13:17:39.812163+0000","last_became_peered":"2024-04-25T13:17:39.812163+0000","last_unstale":"2024-04-25T13:17:42.965750+0000","last_undegraded":"2024-04-25T13:17:42.965750+0000","last_fullsized":"2024-04-25T13:17:42.965750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:20.410328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.816285+0000","last_change":"2024-04-25T13:17:40.829512+0000","last_active":"2024-04-25T13:17:41.816285+0000","last_peered":"2024-04-25T13:17:41.816285+0000","last_clean":"2024-04-25T13:17:41.816285+0000","last_became_active":"2024-04-25T13:17:40.829084+0000","last_became_peered":"2024-04-25T13:17:40.829084+0000","last_unstale":"2024-04-25T13:17:41.816285+0000","last_undegraded":"2024-04-25T13:17:41.816285+0000","last_fullsized":"2024-04-25T13:17:41.816285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:55.473018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821500+0000","last_change":"2024-04-25T13:17:39.810766+0000","last_active":"2024-04-25T13:17:41.821500+0000","last_peered":"2024-04-25T13:17:41.821500+0000","last_clean":"2024-04-25T13:17:41.821500+0000","last_became_active":"2024-04-25T13:17:39.810444+0000","last_became_peered":"2024-04-25T13:17:39.810444+0000","last_unstale":"2024-04-25T13:17:41.821500+0000","last_undegraded":"2024-04-25T13:17:41.821500+0000","last_fullsized":"2024-04-25T13:17:41.821500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:33.470819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.543342+0000","last_change":"2024-04-25T13:17:40.825574+0000","last_active":"2024-04-25T13:17:43.543342+0000","last_peered":"2024-04-25T13:17:43.543342+0000","last_clean":"2024-04-25T13:17:43.543342+0000","last_became_active":"2024-04-25T13:17:40.825220+0000","last_became_peered":"2024-04-25T13:17:40.825220+0000","last_unstale":"2024-04-25T13:17:43.543342+0000","last_undegraded":"2024-04-25T13:17:43.543342+0000","last_fullsized":"2024-04-25T13:17:43.543342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:13.941665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283795+0000","last_change":"2024-04-25T13:17:39.816040+0000","last_active":"2024-04-25T13:17:42.283795+0000","last_peered":"2024-04-25T13:17:42.283795+0000","last_clean":"2024-04-25T13:17:42.283795+0000","last_became_active":"2024-04-25T13:17:39.815746+0000","last_became_peered":"2024-04-25T13:17:39.815746+0000","last_unstale":"2024-04-25T13:17:42.283795+0000","last_undegraded":"2024-04-25T13:17:42.283795+0000","last_fullsized":"2024-04-25T13:17:42.283795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:12.068992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.626722+0000","last_change":"2024-04-25T13:17:40.821926+0000","last_active":"2024-04-25T13:17:42.626722+0000","last_peered":"2024-04-25T13:17:42.626722+0000","last_clean":"2024-04-25T13:17:42.626722+0000","last_became_active":"2024-04-25T13:17:40.821280+0000","last_became_peered":"2024-04-25T13:17:40.821280+0000","last_unstale":"2024-04-25T13:17:42.626722+0000","last_undegraded":"2024-04-25T13:17:42.626722+0000","last_fullsized":"2024-04-25T13:17:42.626722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:33.662553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.966314+0000","last_change":"2024-04-25T13:17:39.819113+0000","last_active":"2024-04-25T13:17:42.966314+0000","last_peered":"2024-04-25T13:17:42.966314+0000","last_clean":"2024-04-25T13:17:42.966314+0000","last_became_active":"2024-04-25T13:17:39.818749+0000","last_became_peered":"2024-04-25T13:17:39.818749+0000","last_unstale":"2024-04-25T13:17:42.966314+0000","last_undegraded":"2024-04-25T13:17:42.966314+0000","last_fullsized":"2024-04-25T13:17:42.966314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:32.119616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.542999+0000","last_change":"2024-04-25T13:17:40.821066+0000","last_active":"2024-04-25T13:17:43.542999+0000","last_peered":"2024-04-25T13:17:43.542999+0000","last_clean":"2024-04-25T13:17:43.542999+0000","last_became_active":"2024-04-25T13:17:40.820674+0000","last_became_peered":"2024-04-25T13:17:40.820674+0000","last_unstale":"2024-04-25T13:17:43.542999+0000","last_undegraded":"2024-04-25T13:17:43.542999+0000","last_fullsized":"2024-04-25T13:17:43.542999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:01.157850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821501+0000","last_change":"2024-04-25T13:17:39.811131+0000","last_active":"2024-04-25T13:17:41.821501+0000","last_peered":"2024-04-25T13:17:41.821501+0000","last_clean":"2024-04-25T13:17:41.821501+0000","last_became_active":"2024-04-25T13:17:39.810791+0000","last_became_peered":"2024-04-25T13:17:39.810791+0000","last_unstale":"2024-04-25T13:17:41.821501+0000","last_undegraded":"2024-04-25T13:17:41.821501+0000","last_fullsized":"2024-04-25T13:17:41.821501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:43.964183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800602+0000","last_change":"2024-04-25T13:17:40.818077+0000","last_active":"2024-04-25T13:17:42.800602+0000","last_peered":"2024-04-25T13:17:42.800602+0000","last_clean":"2024-04-25T13:17:42.800602+0000","last_became_active":"2024-04-25T13:17:40.817757+0000","last_became_peered":"2024-04-25T13:17:40.817757+0000","last_unstale":"2024-04-25T13:17:42.800602+0000","last_undegraded":"2024-04-25T13:17:42.800602+0000","last_fullsized":"2024-04-25T13:17:42.800602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:17.526603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.850721+0000","last_change":"2024-04-25T13:17:39.809111+0000","last_active":"2024-04-25T13:17:41.850721+0000","last_peered":"2024-04-25T13:17:41.850721+0000","last_clean":"2024-04-25T13:17:41.850721+0000","last_became_active":"2024-04-25T13:17:39.808785+0000","last_became_peered":"2024-04-25T13:17:39.808785+0000","last_unstale":"2024-04-25T13:17:41.850721+0000","last_undegraded":"2024-04-25T13:17:41.850721+0000","last_fullsized":"2024-04-25T13:17:41.850721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:41.549940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311564+0000","last_change":"2024-04-25T13:17:40.835646+0000","last_active":"2024-04-25T13:17:43.311564+0000","last_peered":"2024-04-25T13:17:43.311564+0000","last_clean":"2024-04-25T13:17:43.311564+0000","last_became_active":"2024-04-25T13:17:40.835267+0000","last_became_peered":"2024-04-25T13:17:40.835267+0000","last_unstale":"2024-04-25T13:17:43.311564+0000","last_undegraded":"2024-04-25T13:17:43.311564+0000","last_fullsized":"2024-04-25T13:17:43.311564+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:05.811651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.965775+0000","last_change":"2024-04-25T13:17:39.813006+0000","last_active":"2024-04-25T13:17:42.965775+0000","last_peered":"2024-04-25T13:17:42.965775+0000","last_clean":"2024-04-25T13:17:42.965775+0000","last_became_active":"2024-04-25T13:17:39.812635+0000","last_became_peered":"2024-04-25T13:17:39.812635+0000","last_unstale":"2024-04-25T13:17:42.965775+0000","last_undegraded":"2024-04-25T13:17:42.965775+0000","last_fullsized":"2024-04-25T13:17:42.965775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.922568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283851+0000","last_change":"2024-04-25T13:17:40.833365+0000","last_active":"2024-04-25T13:17:42.283851+0000","last_peered":"2024-04-25T13:17:42.283851+0000","last_clean":"2024-04-25T13:17:42.283851+0000","last_became_active":"2024-04-25T13:17:40.832927+0000","last_became_peered":"2024-04-25T13:17:40.832927+0000","last_unstale":"2024-04-25T13:17:42.283851+0000","last_undegraded":"2024-04-25T13:17:42.283851+0000","last_fullsized":"2024-04-25T13:17:42.283851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.622153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.542809+0000","last_change":"2024-04-25T13:17:39.810983+0000","last_active":"2024-04-25T13:17:43.542809+0000","last_peered":"2024-04-25T13:17:43.542809+0000","last_clean":"2024-04-25T13:17:43.542809+0000","last_became_active":"2024-04-25T13:17:39.810366+0000","last_became_peered":"2024-04-25T13:17:39.810366+0000","last_unstale":"2024-04-25T13:17:43.542809+0000","last_undegraded":"2024-04-25T13:17:43.542809+0000","last_fullsized":"2024-04-25T13:17:43.542809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:33.636924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283777+0000","last_change":"2024-04-25T13:17:40.818301+0000","last_active":"2024-04-25T13:17:42.283777+0000","last_peered":"2024-04-25T13:17:42.283777+0000","last_clean":"2024-04-25T13:17:42.283777+0000","last_became_active":"2024-04-25T13:17:40.817903+0000","last_became_peered":"2024-04-25T13:17:40.817903+0000","last_unstale":"2024-04-25T13:17:42.283777+0000","last_undegraded":"2024-04-25T13:17:42.283777+0000","last_fullsized":"2024-04-25T13:17:42.283777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:40.913771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800097+0000","last_change":"2024-04-25T13:17:39.820411+0000","last_active":"2024-04-25T13:17:42.800097+0000","last_peered":"2024-04-25T13:17:42.800097+0000","last_clean":"2024-04-25T13:17:42.800097+0000","last_became_active":"2024-04-25T13:17:39.820051+0000","last_became_peered":"2024-04-25T13:17:39.820051+0000","last_unstale":"2024-04-25T13:17:42.800097+0000","last_undegraded":"2024-04-25T13:17:42.800097+0000","last_fullsized":"2024-04-25T13:17:42.800097+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:08.931024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.966202+0000","last_change":"2024-04-25T13:17:40.828743+0000","last_active":"2024-04-25T13:17:42.966202+0000","last_peered":"2024-04-25T13:17:42.966202+0000","last_clean":"2024-04-25T13:17:42.966202+0000","last_became_active":"2024-04-25T13:17:40.828397+0000","last_became_peered":"2024-04-25T13:17:40.828397+0000","last_unstale":"2024-04-25T13:17:42.966202+0000","last_undegraded":"2024-04-25T13:17:42.966202+0000","last_fullsized":"2024-04-25T13:17:42.966202+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:12.286370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800606+0000","last_change":"2024-04-25T13:17:39.812160+0000","last_active":"2024-04-25T13:17:42.800606+0000","last_peered":"2024-04-25T13:17:42.800606+0000","last_clean":"2024-04-25T13:17:42.800606+0000","last_became_active":"2024-04-25T13:17:39.811846+0000","last_became_peered":"2024-04-25T13:17:39.811846+0000","last_unstale":"2024-04-25T13:17:42.800606+0000","last_undegraded":"2024-04-25T13:17:42.800606+0000","last_fullsized":"2024-04-25T13:17:42.800606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:20.146245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817653+0000","last_change":"2024-04-25T13:17:40.819439+0000","last_active":"2024-04-25T13:17:41.817653+0000","last_peered":"2024-04-25T13:17:41.817653+0000","last_clean":"2024-04-25T13:17:41.817653+0000","last_became_active":"2024-04-25T13:17:40.819003+0000","last_became_peered":"2024-04-25T13:17:40.819003+0000","last_unstale":"2024-04-25T13:17:41.817653+0000","last_undegraded":"2024-04-25T13:17:41.817653+0000","last_fullsized":"2024-04-25T13:17:41.817653+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:43.405033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.816466+0000","last_change":"2024-04-25T13:17:39.819540+0000","last_active":"2024-04-25T13:17:41.816466+0000","last_peered":"2024-04-25T13:17:41.816466+0000","last_clean":"2024-04-25T13:17:41.816466+0000","last_became_active":"2024-04-25T13:17:39.818942+0000","last_became_peered":"2024-04-25T13:17:39.818942+0000","last_unstale":"2024-04-25T13:17:41.816466+0000","last_undegraded":"2024-04-25T13:17:41.816466+0000","last_fullsized":"2024-04-25T13:17:41.816466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:51.300087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821688+0000","last_change":"2024-04-25T13:17:40.818872+0000","last_active":"2024-04-25T13:17:41.821688+0000","last_peered":"2024-04-25T13:17:41.821688+0000","last_clean":"2024-04-25T13:17:41.821688+0000","last_became_active":"2024-04-25T13:17:40.818522+0000","last_became_peered":"2024-04-25T13:17:40.818522+0000","last_unstale":"2024-04-25T13:17:41.821688+0000","last_undegraded":"2024-04-25T13:17:41.821688+0000","last_fullsized":"2024-04-25T13:17:41.821688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.928782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283549+0000","last_change":"2024-04-25T13:17:39.815555+0000","last_active":"2024-04-25T13:17:42.283549+0000","last_peered":"2024-04-25T13:17:42.283549+0000","last_clean":"2024-04-25T13:17:42.283549+0000","last_became_active":"2024-04-25T13:17:39.815249+0000","last_became_peered":"2024-04-25T13:17:39.815249+0000","last_unstale":"2024-04-25T13:17:42.283549+0000","last_undegraded":"2024-04-25T13:17:42.283549+0000","last_fullsized":"2024-04-25T13:17:42.283549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:40.723562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.821493+0000","last_change":"2024-04-25T13:17:40.821643+0000","last_active":"2024-04-25T13:17:41.821493+0000","last_peered":"2024-04-25T13:17:41.821493+0000","last_clean":"2024-04-25T13:17:41.821493+0000","last_became_active":"2024-04-25T13:17:40.820715+0000","last_became_peered":"2024-04-25T13:17:40.820715+0000","last_unstale":"2024-04-25T13:17:41.821493+0000","last_undegraded":"2024-04-25T13:17:41.821493+0000","last_fullsized":"2024-04-25T13:17:41.821493+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:54.046030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.965990+0000","last_change":"2024-04-25T13:17:40.830693+0000","last_active":"2024-04-25T13:17:42.965990+0000","last_peered":"2024-04-25T13:17:42.965990+0000","last_clean":"2024-04-25T13:17:42.965990+0000","last_became_active":"2024-04-25T13:17:40.830298+0000","last_became_peered":"2024-04-25T13:17:40.830298+0000","last_unstale":"2024-04-25T13:17:42.965990+0000","last_undegraded":"2024-04-25T13:17:42.965990+0000","last_fullsized":"2024-04-25T13:17:42.965990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:03.600512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311284+0000","last_change":"2024-04-25T13:17:39.820658+0000","last_active":"2024-04-25T13:17:43.311284+0000","last_peered":"2024-04-25T13:17:43.311284+0000","last_clean":"2024-04-25T13:17:43.311284+0000","last_became_active":"2024-04-25T13:17:39.820122+0000","last_became_peered":"2024-04-25T13:17:39.820122+0000","last_unstale":"2024-04-25T13:17:43.311284+0000","last_undegraded":"2024-04-25T13:17:43.311284+0000","last_fullsized":"2024-04-25T13:17:43.311284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:04.655982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810232+0000","last_change":"2024-04-25T13:17:40.827220+0000","last_active":"2024-04-25T13:17:42.810232+0000","last_peered":"2024-04-25T13:17:42.810232+0000","last_clean":"2024-04-25T13:17:42.810232+0000","last_became_active":"2024-04-25T13:17:40.826517+0000","last_became_peered":"2024-04-25T13:17:40.826517+0000","last_unstale":"2024-04-25T13:17:42.810232+0000","last_undegraded":"2024-04-25T13:17:42.810232+0000","last_fullsized":"2024-04-25T13:17:42.810232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:30.405644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.542955+0000","last_change":"2024-04-25T13:17:39.813893+0000","last_active":"2024-04-25T13:17:43.542955+0000","last_peered":"2024-04-25T13:17:43.542955+0000","last_clean":"2024-04-25T13:17:43.542955+0000","last_became_active":"2024-04-25T13:17:39.812939+0000","last_became_peered":"2024-04-25T13:17:39.812939+0000","last_unstale":"2024-04-25T13:17:43.542955+0000","last_undegraded":"2024-04-25T13:17:43.542955+0000","last_fullsized":"2024-04-25T13:17:43.542955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.033913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283324+0000","last_change":"2024-04-25T13:16:30.734601+0000","last_active":"2024-04-25T13:17:42.283324+0000","last_peered":"2024-04-25T13:17:42.283324+0000","last_clean":"2024-04-25T13:17:42.283324+0000","last_became_active":"2024-04-25T13:16:30.733987+0000","last_became_peered":"2024-04-25T13:16:30.733987+0000","last_unstale":"2024-04-25T13:17:42.283324+0000","last_undegraded":"2024-04-25T13:17:42.283324+0000","last_fullsized":"2024-04-25T13:17:42.283324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:13:02.186391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:13:02.186391+0000","last_clean_scrub_stamp":"2024-04-25T13:13:02.186391+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:41.669205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311424+0000","last_change":"2024-04-25T13:17:40.821671+0000","last_active":"2024-04-25T13:17:43.311424+0000","last_peered":"2024-04-25T13:17:43.311424+0000","last_clean":"2024-04-25T13:17:43.311424+0000","last_became_active":"2024-04-25T13:17:40.821333+0000","last_became_peered":"2024-04-25T13:17:40.821333+0000","last_unstale":"2024-04-25T13:17:43.311424+0000","last_undegraded":"2024-04-25T13:17:43.311424+0000","last_fullsized":"2024-04-25T13:17:43.311424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:50.645335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283939+0000","last_change":"2024-04-25T13:17:39.809530+0000","last_active":"2024-04-25T13:17:42.283939+0000","last_peered":"2024-04-25T13:17:42.283939+0000","last_clean":"2024-04-25T13:17:42.283939+0000","last_became_active":"2024-04-25T13:17:39.809198+0000","last_became_peered":"2024-04-25T13:17:39.809198+0000","last_unstale":"2024-04-25T13:17:42.283939+0000","last_undegraded":"2024-04-25T13:17:42.283939+0000","last_fullsized":"2024-04-25T13:17:42.283939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:34.397912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800427+0000","last_change":"2024-04-25T13:17:39.819607+0000","last_active":"2024-04-25T13:17:42.800427+0000","last_peered":"2024-04-25T13:17:42.800427+0000","last_clean":"2024-04-25T13:17:42.800427+0000","last_became_active":"2024-04-25T13:17:39.819273+0000","last_became_peered":"2024-04-25T13:17:39.819273+0000","last_unstale":"2024-04-25T13:17:42.800427+0000","last_undegraded":"2024-04-25T13:17:42.800427+0000","last_fullsized":"2024-04-25T13:17:42.800427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:39.414085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311723+0000","last_change":"2024-04-25T13:17:40.824554+0000","last_active":"2024-04-25T13:17:43.311723+0000","last_peered":"2024-04-25T13:17:43.311723+0000","last_clean":"2024-04-25T13:17:43.311723+0000","last_became_active":"2024-04-25T13:17:40.824249+0000","last_became_peered":"2024-04-25T13:17:40.824249+0000","last_unstale":"2024-04-25T13:17:43.311723+0000","last_undegraded":"2024-04-25T13:17:43.311723+0000","last_fullsized":"2024-04-25T13:17:43.311723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:17.282312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.816510+0000","last_change":"2024-04-25T13:17:39.820095+0000","last_active":"2024-04-25T13:17:41.816510+0000","last_peered":"2024-04-25T13:17:41.816510+0000","last_clean":"2024-04-25T13:17:41.816510+0000","last_became_active":"2024-04-25T13:17:39.819774+0000","last_became_peered":"2024-04-25T13:17:39.819774+0000","last_unstale":"2024-04-25T13:17:41.816510+0000","last_undegraded":"2024-04-25T13:17:41.816510+0000","last_fullsized":"2024-04-25T13:17:41.816510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:09.154958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810602+0000","last_change":"2024-04-25T13:17:40.830202+0000","last_active":"2024-04-25T13:17:42.810602+0000","last_peered":"2024-04-25T13:17:42.810602+0000","last_clean":"2024-04-25T13:17:42.810602+0000","last_became_active":"2024-04-25T13:17:40.829814+0000","last_became_peered":"2024-04-25T13:17:40.829814+0000","last_unstale":"2024-04-25T13:17:42.810602+0000","last_undegraded":"2024-04-25T13:17:42.810602+0000","last_fullsized":"2024-04-25T13:17:42.810602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:05.492378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.543471+0000","last_change":"2024-04-25T13:17:39.814680+0000","last_active":"2024-04-25T13:17:43.543471+0000","last_peered":"2024-04-25T13:17:43.543471+0000","last_clean":"2024-04-25T13:17:43.543471+0000","last_became_active":"2024-04-25T13:17:39.814324+0000","last_became_peered":"2024-04-25T13:17:39.814324+0000","last_unstale":"2024-04-25T13:17:43.543471+0000","last_undegraded":"2024-04-25T13:17:43.543471+0000","last_fullsized":"2024-04-25T13:17:43.543471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:42.818060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.626572+0000","last_change":"2024-04-25T13:17:40.821020+0000","last_active":"2024-04-25T13:17:42.626572+0000","last_peered":"2024-04-25T13:17:42.626572+0000","last_clean":"2024-04-25T13:17:42.626572+0000","last_became_active":"2024-04-25T13:17:40.820664+0000","last_became_peered":"2024-04-25T13:17:40.820664+0000","last_unstale":"2024-04-25T13:17:42.626572+0000","last_undegraded":"2024-04-25T13:17:42.626572+0000","last_fullsized":"2024-04-25T13:17:42.626572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:27.974369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.542754+0000","last_change":"2024-04-25T13:17:39.820908+0000","last_active":"2024-04-25T13:17:43.542754+0000","last_peered":"2024-04-25T13:17:43.542754+0000","last_clean":"2024-04-25T13:17:43.542754+0000","last_became_active":"2024-04-25T13:17:39.820612+0000","last_became_peered":"2024-04-25T13:17:39.820612+0000","last_unstale":"2024-04-25T13:17:43.542754+0000","last_undegraded":"2024-04-25T13:17:43.542754+0000","last_fullsized":"2024-04-25T13:17:43.542754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:15.170079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800250+0000","last_change":"2024-04-25T13:17:40.817232+0000","last_active":"2024-04-25T13:17:42.800250+0000","last_peered":"2024-04-25T13:17:42.800250+0000","last_clean":"2024-04-25T13:17:42.800250+0000","last_became_active":"2024-04-25T13:17:40.816895+0000","last_became_peered":"2024-04-25T13:17:40.816895+0000","last_unstale":"2024-04-25T13:17:42.800250+0000","last_undegraded":"2024-04-25T13:17:42.800250+0000","last_fullsized":"2024-04-25T13:17:42.800250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:28.772948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.809875+0000","last_change":"2024-04-25T13:17:39.816586+0000","last_active":"2024-04-25T13:17:42.809875+0000","last_peered":"2024-04-25T13:17:42.809875+0000","last_clean":"2024-04-25T13:17:42.809875+0000","last_became_active":"2024-04-25T13:17:39.815712+0000","last_became_peered":"2024-04-25T13:17:39.815712+0000","last_unstale":"2024-04-25T13:17:42.809875+0000","last_undegraded":"2024-04-25T13:17:42.809875+0000","last_fullsized":"2024-04-25T13:17:42.809875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.772789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817304+0000","last_change":"2024-04-25T13:17:40.824505+0000","last_active":"2024-04-25T13:17:41.817304+0000","last_peered":"2024-04-25T13:17:41.817304+0000","last_clean":"2024-04-25T13:17:41.817304+0000","last_became_active":"2024-04-25T13:17:40.824197+0000","last_became_peered":"2024-04-25T13:17:40.824197+0000","last_unstale":"2024-04-25T13:17:41.817304+0000","last_undegraded":"2024-04-25T13:17:41.817304+0000","last_fullsized":"2024-04-25T13:17:41.817304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:17.484269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.850686+0000","last_change":"2024-04-25T13:17:39.810242+0000","last_active":"2024-04-25T13:17:41.850686+0000","last_peered":"2024-04-25T13:17:41.850686+0000","last_clean":"2024-04-25T13:17:41.850686+0000","last_became_active":"2024-04-25T13:17:39.809547+0000","last_became_peered":"2024-04-25T13:17:39.809547+0000","last_unstale":"2024-04-25T13:17:41.850686+0000","last_undegraded":"2024-04-25T13:17:41.850686+0000","last_fullsized":"2024-04-25T13:17:41.850686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.615321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810386+0000","last_change":"2024-04-25T13:17:40.822114+0000","last_active":"2024-04-25T13:17:42.810386+0000","last_peered":"2024-04-25T13:17:42.810386+0000","last_clean":"2024-04-25T13:17:42.810386+0000","last_became_active":"2024-04-25T13:17:40.821517+0000","last_became_peered":"2024-04-25T13:17:40.821517+0000","last_unstale":"2024-04-25T13:17:42.810386+0000","last_undegraded":"2024-04-25T13:17:42.810386+0000","last_fullsized":"2024-04-25T13:17:42.810386+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:07.225119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.800304+0000","last_change":"2024-04-25T13:17:39.812135+0000","last_active":"2024-04-25T13:17:42.800304+0000","last_peered":"2024-04-25T13:17:42.800304+0000","last_clean":"2024-04-25T13:17:42.800304+0000","last_became_active":"2024-04-25T13:17:39.811836+0000","last_became_peered":"2024-04-25T13:17:39.811836+0000","last_unstale":"2024-04-25T13:17:42.800304+0000","last_undegraded":"2024-04-25T13:17:42.800304+0000","last_fullsized":"2024-04-25T13:17:42.800304+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:00.692725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283788+0000","last_change":"2024-04-25T13:17:40.830160+0000","last_active":"2024-04-25T13:17:42.283788+0000","last_peered":"2024-04-25T13:17:42.283788+0000","last_clean":"2024-04-25T13:17:42.283788+0000","last_became_active":"2024-04-25T13:17:40.829773+0000","last_became_peered":"2024-04-25T13:17:40.829773+0000","last_unstale":"2024-04-25T13:17:42.283788+0000","last_undegraded":"2024-04-25T13:17:42.283788+0000","last_fullsized":"2024-04-25T13:17:42.283788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:24.515333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.850910+0000","last_change":"2024-04-25T13:17:39.819011+0000","last_active":"2024-04-25T13:17:41.850910+0000","last_peered":"2024-04-25T13:17:41.850910+0000","last_clean":"2024-04-25T13:17:41.850910+0000","last_became_active":"2024-04-25T13:17:39.818656+0000","last_became_peered":"2024-04-25T13:17:39.818656+0000","last_unstale":"2024-04-25T13:17:41.850910+0000","last_undegraded":"2024-04-25T13:17:41.850910+0000","last_fullsized":"2024-04-25T13:17:41.850910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:20.507500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.284048+0000","last_change":"2024-04-25T13:17:40.829595+0000","last_active":"2024-04-25T13:17:42.284048+0000","last_peered":"2024-04-25T13:17:42.284048+0000","last_clean":"2024-04-25T13:17:42.284048+0000","last_became_active":"2024-04-25T13:17:40.827707+0000","last_became_peered":"2024-04-25T13:17:40.827707+0000","last_unstale":"2024-04-25T13:17:42.284048+0000","last_undegraded":"2024-04-25T13:17:42.284048+0000","last_fullsized":"2024-04-25T13:17:42.284048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:49.002629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.966145+0000","last_change":"2024-04-25T13:17:39.809740+0000","last_active":"2024-04-25T13:17:42.966145+0000","last_peered":"2024-04-25T13:17:42.966145+0000","last_clean":"2024-04-25T13:17:42.966145+0000","last_became_active":"2024-04-25T13:17:39.809407+0000","last_became_peered":"2024-04-25T13:17:39.809407+0000","last_unstale":"2024-04-25T13:17:42.966145+0000","last_undegraded":"2024-04-25T13:17:42.966145+0000","last_fullsized":"2024-04-25T13:17:42.966145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:16.732755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.283716+0000","last_change":"2024-04-25T13:17:40.825573+0000","last_active":"2024-04-25T13:17:42.283716+0000","last_peered":"2024-04-25T13:17:42.283716+0000","last_clean":"2024-04-25T13:17:42.283716+0000","last_became_active":"2024-04-25T13:17:40.825181+0000","last_became_peered":"2024-04-25T13:17:40.825181+0000","last_unstale":"2024-04-25T13:17:42.283716+0000","last_undegraded":"2024-04-25T13:17:42.283716+0000","last_fullsized":"2024-04-25T13:17:42.283716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:06.446494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311533+0000","last_change":"2024-04-25T13:17:39.812458+0000","last_active":"2024-04-25T13:17:43.311533+0000","last_peered":"2024-04-25T13:17:43.311533+0000","last_clean":"2024-04-25T13:17:43.311533+0000","last_became_active":"2024-04-25T13:17:39.812092+0000","last_became_peered":"2024-04-25T13:17:39.812092+0000","last_unstale":"2024-04-25T13:17:43.311533+0000","last_undegraded":"2024-04-25T13:17:43.311533+0000","last_fullsized":"2024-04-25T13:17:43.311533+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:07.210444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.284039+0000","last_change":"2024-04-25T13:17:40.821510+0000","last_active":"2024-04-25T13:17:42.284039+0000","last_peered":"2024-04-25T13:17:42.284039+0000","last_clean":"2024-04-25T13:17:42.284039+0000","last_became_active":"2024-04-25T13:17:40.818897+0000","last_became_peered":"2024-04-25T13:17:40.818897+0000","last_unstale":"2024-04-25T13:17:42.284039+0000","last_undegraded":"2024-04-25T13:17:42.284039+0000","last_fullsized":"2024-04-25T13:17:42.284039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:15.215991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817436+0000","last_change":"2024-04-25T13:17:40.823789+0000","last_active":"2024-04-25T13:17:41.817436+0000","last_peered":"2024-04-25T13:17:41.817436+0000","last_clean":"2024-04-25T13:17:41.817436+0000","last_became_active":"2024-04-25T13:17:40.823424+0000","last_became_peered":"2024-04-25T13:17:40.823424+0000","last_unstale":"2024-04-25T13:17:41.817436+0000","last_undegraded":"2024-04-25T13:17:41.817436+0000","last_fullsized":"2024-04-25T13:17:41.817436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:50.022286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.850693+0000","last_change":"2024-04-25T13:17:40.827085+0000","last_active":"2024-04-25T13:17:41.850693+0000","last_peered":"2024-04-25T13:17:41.850693+0000","last_clean":"2024-04-25T13:17:41.850693+0000","last_became_active":"2024-04-25T13:17:40.826760+0000","last_became_peered":"2024-04-25T13:17:40.826760+0000","last_unstale":"2024-04-25T13:17:41.850693+0000","last_undegraded":"2024-04-25T13:17:41.850693+0000","last_fullsized":"2024-04-25T13:17:41.850693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:46.662788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.850574+0000","last_change":"2024-04-25T13:17:40.824283+0000","last_active":"2024-04-25T13:17:41.850574+0000","last_peered":"2024-04-25T13:17:41.850574+0000","last_clean":"2024-04-25T13:17:41.850574+0000","last_became_active":"2024-04-25T13:17:40.823914+0000","last_became_peered":"2024-04-25T13:17:40.823914+0000","last_unstale":"2024-04-25T13:17:41.850574+0000","last_undegraded":"2024-04-25T13:17:41.850574+0000","last_fullsized":"2024-04-25T13:17:41.850574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:01.158295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.626387+0000","last_change":"2024-04-25T13:17:40.822003+0000","last_active":"2024-04-25T13:17:42.626387+0000","last_peered":"2024-04-25T13:17:42.626387+0000","last_clean":"2024-04-25T13:17:42.626387+0000","last_became_active":"2024-04-25T13:17:40.821453+0000","last_became_peered":"2024-04-25T13:17:40.821453+0000","last_unstale":"2024-04-25T13:17:42.626387+0000","last_undegraded":"2024-04-25T13:17:42.626387+0000","last_fullsized":"2024-04-25T13:17:42.626387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:34.474332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311730+0000","last_change":"2024-04-25T13:17:40.832449+0000","last_active":"2024-04-25T13:17:43.311730+0000","last_peered":"2024-04-25T13:17:43.311730+0000","last_clean":"2024-04-25T13:17:43.311730+0000","last_became_active":"2024-04-25T13:17:40.832079+0000","last_became_peered":"2024-04-25T13:17:40.832079+0000","last_unstale":"2024-04-25T13:17:43.311730+0000","last_undegraded":"2024-04-25T13:17:43.311730+0000","last_fullsized":"2024-04-25T13:17:43.311730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:44.537129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817569+0000","last_change":"2024-04-25T13:17:40.828466+0000","last_active":"2024-04-25T13:17:41.817569+0000","last_peered":"2024-04-25T13:17:41.817569+0000","last_clean":"2024-04-25T13:17:41.817569+0000","last_became_active":"2024-04-25T13:17:40.827996+0000","last_became_peered":"2024-04-25T13:17:40.827996+0000","last_unstale":"2024-04-25T13:17:41.817569+0000","last_undegraded":"2024-04-25T13:17:41.817569+0000","last_fullsized":"2024-04-25T13:17:41.817569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:26.498795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.851157+0000","last_change":"2024-04-25T13:17:40.821024+0000","last_active":"2024-04-25T13:17:41.851157+0000","last_peered":"2024-04-25T13:17:41.851157+0000","last_clean":"2024-04-25T13:17:41.851157+0000","last_became_active":"2024-04-25T13:17:40.820671+0000","last_became_peered":"2024-04-25T13:17:40.820671+0000","last_unstale":"2024-04-25T13:17:41.851157+0000","last_undegraded":"2024-04-25T13:17:41.851157+0000","last_fullsized":"2024-04-25T13:17:41.851157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:57.315116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.810613+0000","last_change":"2024-04-25T13:17:40.837131+0000","last_active":"2024-04-25T13:17:42.810613+0000","last_peered":"2024-04-25T13:17:42.810613+0000","last_clean":"2024-04-25T13:17:42.810613+0000","last_became_active":"2024-04-25T13:17:40.836784+0000","last_became_peered":"2024-04-25T13:17:40.836784+0000","last_unstale":"2024-04-25T13:17:42.810613+0000","last_undegraded":"2024-04-25T13:17:42.810613+0000","last_fullsized":"2024-04-25T13:17:42.810613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:41.818001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.965922+0000","last_change":"2024-04-25T13:17:40.827149+0000","last_active":"2024-04-25T13:17:42.965922+0000","last_peered":"2024-04-25T13:17:42.965922+0000","last_clean":"2024-04-25T13:17:42.965922+0000","last_became_active":"2024-04-25T13:17:40.826800+0000","last_became_peered":"2024-04-25T13:17:40.826800+0000","last_unstale":"2024-04-25T13:17:42.965922+0000","last_undegraded":"2024-04-25T13:17:42.965922+0000","last_fullsized":"2024-04-25T13:17:42.965922+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:46.211785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:42.809855+0000","last_change":"2024-04-25T13:17:40.825028+0000","last_active":"2024-04-25T13:17:42.809855+0000","last_peered":"2024-04-25T13:17:42.809855+0000","last_clean":"2024-04-25T13:17:42.809855+0000","last_became_active":"2024-04-25T13:17:40.824614+0000","last_became_peered":"2024-04-25T13:17:40.824614+0000","last_unstale":"2024-04-25T13:17:42.809855+0000","last_undegraded":"2024-04-25T13:17:42.809855+0000","last_fullsized":"2024-04-25T13:17:42.809855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:07.629080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.542763+0000","last_change":"2024-04-25T13:17:40.836345+0000","last_active":"2024-04-25T13:17:43.542763+0000","last_peered":"2024-04-25T13:17:43.542763+0000","last_clean":"2024-04-25T13:17:43.542763+0000","last_became_active":"2024-04-25T13:17:40.835989+0000","last_became_peered":"2024-04-25T13:17:40.835989+0000","last_unstale":"2024-04-25T13:17:43.542763+0000","last_undegraded":"2024-04-25T13:17:43.542763+0000","last_fullsized":"2024-04-25T13:17:43.542763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:09.418328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:43.311818+0000","last_change":"2024-04-25T13:17:40.835190+0000","last_active":"2024-04-25T13:17:43.311818+0000","last_peered":"2024-04-25T13:17:43.311818+0000","last_clean":"2024-04-25T13:17:43.311818+0000","last_became_active":"2024-04-25T13:17:40.834847+0000","last_became_peered":"2024-04-25T13:17:40.834847+0000","last_unstale":"2024-04-25T13:17:43.311818+0000","last_undegraded":"2024-04-25T13:17:43.311818+0000","last_fullsized":"2024-04-25T13:17:43.311818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:18.217464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T13:17:41.817389+0000","last_change":"2024-04-25T13:17:40.827578+0000","last_active":"2024-04-25T13:17:41.817389+0000","last_peered":"2024-04-25T13:17:41.817389+0000","last_clean":"2024-04-25T13:17:41.817389+0000","last_became_active":"2024-04-25T13:17:40.827256+0000","last_became_peered":"2024-04-25T13:17:40.827256+0000","last_unstale":"2024-04-25T13:17:41.817389+0000","last_undegraded":"2024-04-25T13:17:41.817389+0000","last_fullsized":"2024-04-25T13:17:41.817389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:40.738629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":63094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285581,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":451857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547217,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36868,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708284,"statfs":{"total":95995035648,"available":95957282816,"internally_reserved":0,"allocated":1560576,"data_stored":1044031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743448,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":452439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":950272,"data_stored":452461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266721,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1540096,"data_stored":1042807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":443287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":472752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":451947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":472786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117047,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":950272,"data_stored":451961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":933888,"data_stored":443309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36852,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708300,"statfs":{"total":95995035648,"available":95957299200,"internally_reserved":0,"allocated":1552384,"data_stored":1042524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":20929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:17:48.011 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:17:48.012 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T13:17:48.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T13:17:48.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:47 smithi066 ceph-mon[33137]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:48.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:47 smithi066 ceph-mon[33137]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:17:48.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:47 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/910220169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:48.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:47 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2127621898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:48.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:47 smithi136 ceph-mon[33038]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:48.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:47 smithi136 ceph-mon[33038]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:17:48.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:47 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/910220169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:17:48.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:47 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2127621898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:48.292 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:48 smithi066 ceph-mon[33137]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:17:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:48 smithi066 ceph-mon[33137]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-25T13:17:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:48 smithi066 ceph-mon[33137]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:17:49.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:48 smithi136 ceph-mon[33038]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:17:49.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:48 smithi136 ceph-mon[33038]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-25T13:17:49.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:48 smithi136 ceph-mon[33038]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:17:49.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:48 smithi031 ceph-mon[29457]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:17:49.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:48 smithi031 ceph-mon[29457]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-25T13:17:49.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:48 smithi031 ceph-mon[29457]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:17:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.873+0000 7f92d7350700 1 -- 172.21.15.31:0/116740904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d0101c30 msgr2=0x7f92d01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.873+0000 7f92d7350700 1 --2- 172.21.15.31:0/116740904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d0101c30 0x7f92d01020b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009990 tx=0x7f92c0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:17:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.873+0000 7f92d7350700 1 -- 172.21.15.31:0/116740904 shutdown_connections 2024-04-25T13:17:49.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.873+0000 7f92d7350700 1 --2- 172.21.15.31:0/116740904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d0103060 0x7f92d0103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:49.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.873+0000 7f92d7350700 1 --2- 172.21.15.31:0/116740904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d01025f0 0x7f92d0109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:49.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.873+0000 7f92d7350700 1 --2- 172.21.15.31:0/116740904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d0101c30 0x7f92d01020b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:49.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.873+0000 7f92d7350700 1 -- 172.21.15.31:0/116740904 >> 172.21.15.31:0/116740904 conn(0x7f92d00fc700 msgr2=0x7f92d00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:49.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.873+0000 7f92d7350700 1 -- 172.21.15.31:0/116740904 shutdown_connections 2024-04-25T13:17:49.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.873+0000 7f92d7350700 1 -- 172.21.15.31:0/116740904 wait complete. 2024-04-25T13:17:49.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.873+0000 7f92d7350700 1 Processor -- start 2024-04-25T13:17:49.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d7350700 1 -- start start 2024-04-25T13:17:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d7350700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d0101c30 0x7f92d0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:49.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d7350700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d01025f0 0x7f92d0199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d7350700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d0103060 0x7f92d01a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d7350700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f92d010d4d0 con 0x7f92d0103060 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d7350700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f92d010d350 con 0x7f92d0101c30 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d7350700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f92d010d650 con 0x7f92d01025f0 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d58ed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d0103060 0x7f92d01a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d58ed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d0103060 0x7f92d01a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50936/0 (socket says 172.21.15.31:50936) 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d58ed700 1 -- 172.21.15.31:0/3965602396 learned_addr learned my addr 172.21.15.31:0/3965602396 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d50ec700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d0101c30 0x7f92d0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d48eb700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d01025f0 0x7f92d0199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d58ed700 1 -- 172.21.15.31:0/3965602396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d01025f0 msgr2=0x7f92d0199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d58ed700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d01025f0 0x7f92d0199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d58ed700 1 -- 172.21.15.31:0/3965602396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d0101c30 msgr2=0x7f92d0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d58ed700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d0101c30 0x7f92d0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d58ed700 1 -- 172.21.15.31:0/3965602396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d01a0820 con 0x7f92d0103060 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d50ec700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d0101c30 0x7f92d0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:49.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d48eb700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d01025f0 0x7f92d0199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:49.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.874+0000 7f92d58ed700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d0103060 0x7f92d01a0180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92c8007f10 tx=0x7f92c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:49.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.875+0000 7f92c67fc700 1 -- 172.21.15.31:0/3965602396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8004bb0 con 0x7f92d0103060 2024-04-25T13:17:49.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.875+0000 7f92d7350700 1 -- 172.21.15.31:0/3965602396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d01a0b10 con 0x7f92d0103060 2024-04-25T13:17:49.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.875+0000 7f92c67fc700 1 -- 172.21.15.31:0/3965602396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c8004d50 con 0x7f92d0103060 2024-04-25T13:17:49.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.875+0000 7f92c67fc700 1 -- 172.21.15.31:0/3965602396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8005a30 con 0x7f92d0103060 2024-04-25T13:17:49.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.875+0000 7f92d7350700 1 -- 172.21.15.31:0/3965602396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d01a1050 con 0x7f92d0103060 2024-04-25T13:17:49.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.876+0000 7f92d7350700 1 -- 172.21.15.31:0/3965602396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d0050f80 con 0x7f92d0103060 2024-04-25T13:17:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.877+0000 7f92c67fc700 1 -- 172.21.15.31:0/3965602396 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f92c801a020 con 0x7f92d0103060 2024-04-25T13:17:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.877+0000 7f92c67fc700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f92bc0a3d80 0x7f92bc0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.877+0000 7f92d50ec700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f92bc0a3d80 0x7f92bc0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.877+0000 7f92c67fc700 1 -- 172.21.15.31:0/3965602396 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f92c80cfc60 con 0x7f92d0103060 2024-04-25T13:17:49.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.878+0000 7f92d50ec700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f92bc0a3d80 0x7f92bc0a6240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92cc007900 tx=0x7f92cc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:49.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:49.882+0000 7f92c67fc700 1 -- 172.21.15.31:0/3965602396 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92c8010070 con 0x7f92d0103060 2024-04-25T13:17:50.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.177+0000 7f92d7350700 1 -- 172.21.15.31:0/3965602396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f92d01a1580 con 0x7f92d0103060 2024-04-25T13:17:50.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.179+0000 7f92c67fc700 1 -- 172.21.15.31:0/3965602396 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f92d01a1580 con 0x7f92d0103060 2024-04-25T13:17:50.180 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-04-25T13:17:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQD+Vypm3T2vChAAkoWSbxajBKqkv6IUw11okg== 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 -- 172.21.15.31:0/3965602396 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f92bc0a3d80 msgr2=0x7f92bc0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f92bc0a3d80 0x7f92bc0a6240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f92cc007900 tx=0x7f92cc008040 comp rx=0 tx=0).stop 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 -- 172.21.15.31:0/3965602396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d0103060 msgr2=0x7f92d01a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d0103060 0x7f92d01a0180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92c8007f10 tx=0x7f92c800d3b0 comp rx=0 tx=0).stop 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 -- 172.21.15.31:0/3965602396 shutdown_connections 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92d0103060 0x7f92d01a0180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d0101c30 0x7f92d0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f92bc0a3d80 0x7f92bc0a6240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 --2- 172.21.15.31:0/3965602396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92d01025f0 0x7f92d0199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 -- 172.21.15.31:0/3965602396 >> 172.21.15.31:0/3965602396 conn(0x7f92d00fc700 msgr2=0x7f92d00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 -- 172.21.15.31:0/3965602396 shutdown_connections 2024-04-25T13:17:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:50.182+0000 7f92d7350700 1 -- 172.21.15.31:0/3965602396 wait complete. 2024-04-25T13:17:50.792 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:17:50.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T13:17:50.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T13:17:50.830 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T13:17:50.830 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T13:17:50.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T13:17:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:51.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:50 smithi066 ceph-mon[33137]: pgmap v297: 97 pgs: 97 active+clean; 620 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-04-25T13:17:51.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:50 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3965602396' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:17:51.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:50 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3965602396' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:17:51.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:50 smithi136 ceph-mon[33038]: pgmap v297: 97 pgs: 97 active+clean; 620 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-04-25T13:17:51.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:50 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3965602396' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:17:51.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:50 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3965602396' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:17:51.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:50 smithi031 ceph-mon[29457]: pgmap v297: 97 pgs: 97 active+clean; 620 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2024-04-25T13:17:51.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:50 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3965602396' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:17:51.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:50 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3965602396' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:17:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.733+0000 7f0e839f4700 1 -- 172.21.15.31:0/1892554589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c108a70 msgr2=0x7f0e7c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.733+0000 7f0e839f4700 1 --2- 172.21.15.31:0/1892554589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c108a70 0x7f0e7c108e50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009990 tx=0x7f0e78009ca0 comp rx=0 tx=0).stop 2024-04-25T13:17:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.733+0000 7f0e839f4700 1 -- 172.21.15.31:0/1892554589 shutdown_connections 2024-04-25T13:17:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.733+0000 7f0e839f4700 1 --2- 172.21.15.31:0/1892554589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e7c103480 0x7f0e7c10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.733+0000 7f0e839f4700 1 --2- 172.21.15.31:0/1892554589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c108a70 0x7f0e7c108e50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.733+0000 7f0e839f4700 1 --2- 172.21.15.31:0/1892554589 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e7c102ac0 0x7f0e7c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:52.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.733+0000 7f0e839f4700 1 -- 172.21.15.31:0/1892554589 >> 172.21.15.31:0/1892554589 conn(0x7f0e7c079bb0 msgr2=0x7f0e7c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.733+0000 7f0e839f4700 1 -- 172.21.15.31:0/1892554589 shutdown_connections 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.733+0000 7f0e839f4700 1 -- 172.21.15.31:0/1892554589 wait complete. 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e839f4700 1 Processor -- start 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e839f4700 1 -- start start 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e839f4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e7c102ac0 0x7f0e7c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e839f4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c103480 0x7f0e7c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e839f4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e7c108a70 0x7f0e7c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e839f4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c112c80 con 0x7f0e7c103480 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e839f4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c112b00 con 0x7f0e7c108a70 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e839f4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c112e00 con 0x7f0e7c102ac0 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e80f8f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c103480 0x7f0e7c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e80f8f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c103480 0x7f0e7c075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40118/0 (socket says 172.21.15.31:40118) 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e80f8f700 1 -- 172.21.15.31:0/3562576287 learned_addr learned my addr 172.21.15.31:0/3562576287 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e81790700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e7c102ac0 0x7f0e7c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:52.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e81f91700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e7c108a70 0x7f0e7c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.734+0000 7f0e80f8f700 1 -- 172.21.15.31:0/3562576287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e7c102ac0 msgr2=0x7f0e7c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.735+0000 7f0e80f8f700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e7c102ac0 0x7f0e7c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.735+0000 7f0e80f8f700 1 -- 172.21.15.31:0/3562576287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e7c108a70 msgr2=0x7f0e7c0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.735+0000 7f0e80f8f700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e7c108a70 0x7f0e7c0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.735+0000 7f0e80f8f700 1 -- 172.21.15.31:0/3562576287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c1a7600 con 0x7f0e7c103480 2024-04-25T13:17:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.735+0000 7f0e80f8f700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c103480 0x7f0e7c075280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c9b0 tx=0x7f0e6c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.735+0000 7f0e727fc700 1 -- 172.21.15.31:0/3562576287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c007b00 con 0x7f0e7c103480 2024-04-25T13:17:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.735+0000 7f0e839f4700 1 -- 172.21.15.31:0/3562576287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c1a78f0 con 0x7f0e7c103480 2024-04-25T13:17:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.735+0000 7f0e727fc700 1 -- 172.21.15.31:0/3562576287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6c00ce50 con 0x7f0e7c103480 2024-04-25T13:17:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.735+0000 7f0e727fc700 1 -- 172.21.15.31:0/3562576287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c018660 con 0x7f0e7c103480 2024-04-25T13:17:52.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.735+0000 7f0e839f4700 1 -- 172.21.15.31:0/3562576287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c1a7e80 con 0x7f0e7c103480 2024-04-25T13:17:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.736+0000 7f0e839f4700 1 -- 172.21.15.31:0/3562576287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e7c069730 con 0x7f0e7c103480 2024-04-25T13:17:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.737+0000 7f0e727fc700 1 -- 172.21.15.31:0/3562576287 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0e6c00f450 con 0x7f0e7c103480 2024-04-25T13:17:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.738+0000 7f0e727fc700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e680a3d50 0x7f0e680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.738+0000 7f0e81790700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e680a3d50 0x7f0e680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.738+0000 7f0e727fc700 1 -- 172.21.15.31:0/3562576287 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f0e6c0d0530 con 0x7f0e7c103480 2024-04-25T13:17:52.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.738+0000 7f0e81790700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e680a3d50 0x7f0e680a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800b5c0 tx=0x7f0e78005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:52.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:52.741+0000 7f0e727fc700 1 -- 172.21.15.31:0/3562576287 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e6c014030 con 0x7f0e7c103480 2024-04-25T13:17:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.012+0000 7f0e839f4700 1 -- 172.21.15.31:0/3562576287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0e7c02e020 con 0x7f0e7c103480 2024-04-25T13:17:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.017+0000 7f0e727fc700 1 -- 172.21.15.31:0/3562576287 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264994 (secure 0 0 0) 0x7f0e6c09d070 con 0x7f0e7c103480 2024-04-25T13:17:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 -- 172.21.15.31:0/3562576287 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e680a3d50 msgr2=0x7f0e680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e680a3d50 0x7f0e680a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800b5c0 tx=0x7f0e78005960 comp rx=0 tx=0).stop 2024-04-25T13:17:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 -- 172.21.15.31:0/3562576287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c103480 msgr2=0x7f0e7c075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c103480 0x7f0e7c075280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c9b0 tx=0x7f0e6c00ccc0 comp rx=0 tx=0).stop 2024-04-25T13:17:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 -- 172.21.15.31:0/3562576287 shutdown_connections 2024-04-25T13:17:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0e680a3d50 0x7f0e680a6210 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e7c102ac0 0x7f0e7c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0e7c108a70 0x7f0e7c0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 --2- 172.21.15.31:0/3562576287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c103480 0x7f0e7c075280 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 -- 172.21.15.31:0/3562576287 >> 172.21.15.31:0/3562576287 conn(0x7f0e7c079bb0 msgr2=0x7f0e7c1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 -- 172.21.15.31:0/3562576287 shutdown_connections 2024-04-25T13:17:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:53.020+0000 7f0e839f4700 1 -- 172.21.15.31:0/3562576287 wait complete. 2024-04-25T13:17:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:52 smithi066 ceph-mon[33137]: pgmap v298: 97 pgs: 97 active+clean; 620 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-25T13:17:53.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:52 smithi136 ceph-mon[33038]: pgmap v298: 97 pgs: 97 active+clean; 620 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-25T13:17:53.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:52 smithi031 ceph-mon[29457]: pgmap v298: 97 pgs: 97 active+clean; 620 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-25T13:17:53.596 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":958678691},{"type":"v1","addr":"172.21.15.31:6801","nonce":958678691}]},"active_addr":"172.21.15.31:6801/958678691","active_change":"2024-04-25T13:10:31.702758+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1259986822}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2022600806}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":996243284}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2153915817}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1237462528}]}]} 2024-04-25T13:17:53.602 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T13:17:53.603 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T13:17:53.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:17:53.854 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:54.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:53 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3562576287' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:17:54.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:53 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3562576287' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:17:54.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:53 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3562576287' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:17:55.144 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:54 smithi031 ceph-mon[29457]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-04-25T13:17:55.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:54 smithi066 ceph-mon[33137]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-04-25T13:17:55.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:54 smithi136 ceph-mon[33038]: pgmap v299: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2024-04-25T13:17:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.374+0000 7f89de1a6700 1 -- 172.21.15.31:0/710571347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8101940 msgr2=0x7f89d8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.374+0000 7f89de1a6700 1 --2- 172.21.15.31:0/710571347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8101940 0x7f89d8101dc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009990 tx=0x7f89c8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:17:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.374+0000 7f89de1a6700 1 -- 172.21.15.31:0/710571347 shutdown_connections 2024-04-25T13:17:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.374+0000 7f89de1a6700 1 --2- 172.21.15.31:0/710571347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d8107990 0x7f89d8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:55.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.374+0000 7f89de1a6700 1 --2- 172.21.15.31:0/710571347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89d8102300 0x7f89d810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:55.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.374+0000 7f89de1a6700 1 --2- 172.21.15.31:0/710571347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8101940 0x7f89d8101dc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:55.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.374+0000 7f89de1a6700 1 -- 172.21.15.31:0/710571347 >> 172.21.15.31:0/710571347 conn(0x7f89d80788f0 msgr2=0x7f89d8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:55.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.374+0000 7f89de1a6700 1 -- 172.21.15.31:0/710571347 shutdown_connections 2024-04-25T13:17:55.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.374+0000 7f89de1a6700 1 -- 172.21.15.31:0/710571347 wait complete. 2024-04-25T13:17:55.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.375+0000 7f89de1a6700 1 Processor -- start 2024-04-25T13:17:55.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.375+0000 7f89de1a6700 1 -- start start 2024-04-25T13:17:55.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.375+0000 7f89de1a6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8101940 0x7f89d81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:55.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.375+0000 7f89de1a6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89d8102300 0x7f89d8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:55.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.375+0000 7f89de1a6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d8107990 0x7f89d819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:55.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.375+0000 7f89de1a6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f89d8111b60 con 0x7f89d8101940 2024-04-25T13:17:55.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.375+0000 7f89de1a6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f89d81119e0 con 0x7f89d8102300 2024-04-25T13:17:55.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.375+0000 7f89de1a6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f89d8111ce0 con 0x7f89d8107990 2024-04-25T13:17:55.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.375+0000 7f89d77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8101940 0x7f89d81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:55.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.375+0000 7f89d77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8101940 0x7f89d81991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40134/0 (socket says 172.21.15.31:40134) 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.375+0000 7f89d77fe700 1 -- 172.21.15.31:0/3848389188 learned_addr learned my addr 172.21.15.31:0/3848389188 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d6ffd700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89d8102300 0x7f89d8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d7fff700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d8107990 0x7f89d819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d77fe700 1 -- 172.21.15.31:0/3848389188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d8107990 msgr2=0x7f89d819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d77fe700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d8107990 0x7f89d819da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d77fe700 1 -- 172.21.15.31:0/3848389188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89d8102300 msgr2=0x7f89d8199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d77fe700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89d8102300 0x7f89d8199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d77fe700 1 -- 172.21.15.31:0/3848389188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d819e1a0 con 0x7f89d8101940 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d6ffd700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89d8102300 0x7f89d8199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d77fe700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8101940 0x7f89d81991e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89c000b530 tx=0x7f89c000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d4ff9700 1 -- 172.21.15.31:0/3848389188 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0015020 con 0x7f89d8101940 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d4ff9700 1 -- 172.21.15.31:0/3848389188 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c0011ce0 con 0x7f89d8101940 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89de1a6700 1 -- 172.21.15.31:0/3848389188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d819e490 con 0x7f89d8101940 2024-04-25T13:17:55.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89d4ff9700 1 -- 172.21.15.31:0/3848389188 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0010d20 con 0x7f89d8101940 2024-04-25T13:17:55.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.376+0000 7f89de1a6700 1 -- 172.21.15.31:0/3848389188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d81a5d10 con 0x7f89d8101940 2024-04-25T13:17:55.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.377+0000 7f89de1a6700 1 -- 172.21.15.31:0/3848389188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d8069730 con 0x7f89d8101940 2024-04-25T13:17:55.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.378+0000 7f89d4ff9700 1 -- 172.21.15.31:0/3848389188 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f89c0018020 con 0x7f89d8101940 2024-04-25T13:17:55.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.379+0000 7f89d4ff9700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f89c40a3d50 0x7f89c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:55.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.379+0000 7f89d6ffd700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f89c40a3d50 0x7f89c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:55.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.379+0000 7f89d4ff9700 1 -- 172.21.15.31:0/3848389188 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f89c00cfbf0 con 0x7f89d8101940 2024-04-25T13:17:55.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.379+0000 7f89d6ffd700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f89c40a3d50 0x7f89c40a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f89c8006010 tx=0x7f89c800b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:55.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.384+0000 7f89d4ff9700 1 -- 172.21.15.31:0/3848389188 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89c001f020 con 0x7f89d8101940 2024-04-25T13:17:55.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.593+0000 7f89de1a6700 1 -- 172.21.15.31:0/3848389188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89d819a4f0 con 0x7f89d8101940 2024-04-25T13:17:55.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.595+0000 7f89d4ff9700 1 -- 172.21.15.31:0/3848389188 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22182 (secure 0 0 0) 0x7f89c009c630 con 0x7f89d8101940 2024-04-25T13:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:55.596 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:17:41.809574+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:17:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 -- 172.21.15.31:0/3848389188 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f89c40a3d50 msgr2=0x7f89c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f89c40a3d50 0x7f89c40a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f89c8006010 tx=0x7f89c800b540 comp rx=0 tx=0).stop 2024-04-25T13:17:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 -- 172.21.15.31:0/3848389188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8101940 msgr2=0x7f89d81991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8101940 0x7f89d81991e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89c000b530 tx=0x7f89c000b8f0 comp rx=0 tx=0).stop 2024-04-25T13:17:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 -- 172.21.15.31:0/3848389188 shutdown_connections 2024-04-25T13:17:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f89c40a3d50 0x7f89c40a6210 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d8107990 0x7f89d819da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:55.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8101940 0x7f89d81991e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:55.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 --2- 172.21.15.31:0/3848389188 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89d8102300 0x7f89d8199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:55.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 -- 172.21.15.31:0/3848389188 >> 172.21.15.31:0/3848389188 conn(0x7f89d80788f0 msgr2=0x7f89d806ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:55.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 -- 172.21.15.31:0/3848389188 shutdown_connections 2024-04-25T13:17:55.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:55.597+0000 7f89de1a6700 1 -- 172.21.15.31:0/3848389188 wait complete. 2024-04-25T13:17:56.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:55 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3848389188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:56.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:55 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3848389188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:56.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:55 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3848389188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:56.285 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T13:17:56.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:17:56.548 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:56 smithi066 ceph-mon[33137]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-25T13:17:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:56 smithi136 ceph-mon[33038]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-25T13:17:57.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:56 smithi031 ceph-mon[29457]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-25T13:17:58.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.078+0000 7efe00a21700 1 -- 172.21.15.31:0/1260610031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdfc108ab0 msgr2=0x7efdfc108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:58.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.078+0000 7efe00a21700 1 --2- 172.21.15.31:0/1260610031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdfc108ab0 0x7efdfc108e90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efdec009990 tx=0x7efdec009ca0 comp rx=0 tx=0).stop 2024-04-25T13:17:58.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.078+0000 7efe00a21700 1 -- 172.21.15.31:0/1260610031 shutdown_connections 2024-04-25T13:17:58.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.078+0000 7efe00a21700 1 --2- 172.21.15.31:0/1260610031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdfc103480 0x7efdfc10f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:58.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.078+0000 7efe00a21700 1 --2- 172.21.15.31:0/1260610031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdfc108ab0 0x7efdfc108e90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:58.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.078+0000 7efe00a21700 1 --2- 172.21.15.31:0/1260610031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc102ac0 0x7efdfc102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:58.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.078+0000 7efe00a21700 1 -- 172.21.15.31:0/1260610031 >> 172.21.15.31:0/1260610031 conn(0x7efdfc079bf0 msgr2=0x7efdfc07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:58.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.078+0000 7efe00a21700 1 -- 172.21.15.31:0/1260610031 shutdown_connections 2024-04-25T13:17:58.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.078+0000 7efe00a21700 1 -- 172.21.15.31:0/1260610031 wait complete. 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efe00a21700 1 Processor -- start 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efe00a21700 1 -- start start 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efe00a21700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdfc102ac0 0x7efdfc199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efe00a21700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc103480 0x7efdfc199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efe00a21700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdfc108ab0 0x7efdfc19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efe00a21700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc112cf0 con 0x7efdfc108ab0 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efe00a21700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc112b70 con 0x7efdfc103480 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efe00a21700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc112e70 con 0x7efdfc102ac0 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efdfad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdfc108ab0 0x7efdfc19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efdfad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdfc108ab0 0x7efdfc19da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40158/0 (socket says 172.21.15.31:40158) 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efdfad9d700 1 -- 172.21.15.31:0/2844100033 learned_addr learned my addr 172.21.15.31:0/2844100033 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.079+0000 7efdfa59c700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdfc102ac0 0x7efdfc199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:58.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efdf3fff700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc103480 0x7efdfc199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efdfad9d700 1 -- 172.21.15.31:0/2844100033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdfc102ac0 msgr2=0x7efdfc199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efdfad9d700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdfc102ac0 0x7efdfc199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efdfad9d700 1 -- 172.21.15.31:0/2844100033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc103480 msgr2=0x7efdfc199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efdfad9d700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc103480 0x7efdfc199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efdfad9d700 1 -- 172.21.15.31:0/2844100033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc19e150 con 0x7efdfc108ab0 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efdf3fff700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc103480 0x7efdfc199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efdfad9d700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdfc108ab0 0x7efdfc19da10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efdf4007ee0 tx=0x7efdf400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efdf37fe700 1 -- 172.21.15.31:0/2844100033 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf400dbd0 con 0x7efdfc108ab0 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efdf37fe700 1 -- 172.21.15.31:0/2844100033 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf400dd70 con 0x7efdfc108ab0 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efe00a21700 1 -- 172.21.15.31:0/2844100033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc19e440 con 0x7efdfc108ab0 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efdf37fe700 1 -- 172.21.15.31:0/2844100033 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4005990 con 0x7efdfc108ab0 2024-04-25T13:17:58.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.080+0000 7efe00a21700 1 -- 172.21.15.31:0/2844100033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc1a5cc0 con 0x7efdfc108ab0 2024-04-25T13:17:58.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.081+0000 7efe00a21700 1 -- 172.21.15.31:0/2844100033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdfc1041f0 con 0x7efdfc108ab0 2024-04-25T13:17:58.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.082+0000 7efdf37fe700 1 -- 172.21.15.31:0/2844100033 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efdf401a020 con 0x7efdfc108ab0 2024-04-25T13:17:58.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.083+0000 7efdf37fe700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efde80a3cd0 0x7efde80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:58.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.083+0000 7efdfa59c700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efde80a3cd0 0x7efde80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:58.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.083+0000 7efdf37fe700 1 -- 172.21.15.31:0/2844100033 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7efdf40cfaa0 con 0x7efdfc108ab0 2024-04-25T13:17:58.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.083+0000 7efdfa59c700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efde80a3cd0 0x7efde80a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efdec00b5c0 tx=0x7efdec005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:58.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.088+0000 7efdf37fe700 1 -- 172.21.15.31:0/2844100033 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efdf4010070 con 0x7efdfc108ab0 2024-04-25T13:17:58.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.298+0000 7efe00a21700 1 -- 172.21.15.31:0/2844100033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdfc02df80 con 0x7efdfc108ab0 2024-04-25T13:17:58.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.300+0000 7efdf37fe700 1 -- 172.21.15.31:0/2844100033 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22182 (secure 0 0 0) 0x7efdf409c4d0 con 0x7efdfc108ab0 2024-04-25T13:17:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:17:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:17:41.809574+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:17:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 -- 172.21.15.31:0/2844100033 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efde80a3cd0 msgr2=0x7efde80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efde80a3cd0 0x7efde80a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efdec00b5c0 tx=0x7efdec005960 comp rx=0 tx=0).stop 2024-04-25T13:17:58.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 -- 172.21.15.31:0/2844100033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdfc108ab0 msgr2=0x7efdfc19da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:58.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdfc108ab0 0x7efdfc19da10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efdf4007ee0 tx=0x7efdf400d3b0 comp rx=0 tx=0).stop 2024-04-25T13:17:58.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 -- 172.21.15.31:0/2844100033 shutdown_connections 2024-04-25T13:17:58.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efde80a3cd0 0x7efde80a6190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:58.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdfc102ac0 0x7efdfc199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:58.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdfc108ab0 0x7efdfc19da10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 --2- 172.21.15.31:0/2844100033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efdfc103480 0x7efdfc199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 -- 172.21.15.31:0/2844100033 >> 172.21.15.31:0/2844100033 conn(0x7efdfc079bf0 msgr2=0x7efdfc100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 -- 172.21.15.31:0/2844100033 shutdown_connections 2024-04-25T13:17:58.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:17:58.302+0000 7efe00a21700 1 -- 172.21.15.31:0/2844100033 wait complete. 2024-04-25T13:17:59.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T13:17:59.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T13:17:59.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T13:17:59.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T13:17:59.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T13:17:59.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T13:17:59.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T13:17:59.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T13:17:59.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T13:17:59.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T13:17:59.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T13:17:59.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T13:17:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:58 smithi066 ceph-mon[33137]: pgmap v301: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-04-25T13:17:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:17:58 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2844100033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:58 smithi136 ceph-mon[33038]: pgmap v301: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-04-25T13:17:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:17:58 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2844100033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:59.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:58 smithi031 ceph-mon[29457]: pgmap v301: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-04-25T13:17:59.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:17:58 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2844100033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:17:59.665 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:59.672 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:59.684 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:59.965 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:59.965 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:59.966 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:01.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:00 smithi031 ceph-mon[29457]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-25T13:18:01.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:00 smithi066 ceph-mon[33137]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-25T13:18:01.209 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:00 smithi136 ceph-mon[33038]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-25T13:18:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:02 smithi066 ceph-mon[33137]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T13:18:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:02 smithi136 ceph-mon[33038]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T13:18:03.232 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:02 smithi031 ceph-mon[29457]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T13:18:05.152 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:04 smithi031 ceph-mon[29457]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T13:18:05.157 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:04 smithi066 ceph-mon[33137]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T13:18:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:04 smithi136 ceph-mon[33038]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T13:18:06.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.129+0000 7facfd9aa700 1 -- 172.21.15.31:0/865701010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf8108a70 msgr2=0x7facf8108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:06.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.129+0000 7facfd9aa700 1 --2- 172.21.15.31:0/865701010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf8108a70 0x7facf8108e50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7face8009990 tx=0x7face8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:06.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.129+0000 7facfd9aa700 1 -- 172.21.15.31:0/865701010 shutdown_connections 2024-04-25T13:18:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.129+0000 7facfd9aa700 1 --2- 172.21.15.31:0/865701010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf8103480 0x7facf810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.129+0000 7facfd9aa700 1 --2- 172.21.15.31:0/865701010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf8108a70 0x7facf8108e50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.129+0000 7facfd9aa700 1 --2- 172.21.15.31:0/865701010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf8102ac0 0x7facf8102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.129+0000 7facfd9aa700 1 -- 172.21.15.31:0/865701010 >> 172.21.15.31:0/865701010 conn(0x7facf8079bb0 msgr2=0x7facf8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.129+0000 7facfd9aa700 1 -- 172.21.15.31:0/865701010 shutdown_connections 2024-04-25T13:18:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.129+0000 7facfd9aa700 1 -- 172.21.15.31:0/865701010 wait complete. 2024-04-25T13:18:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.129+0000 7facfd9aa700 1 Processor -- start 2024-04-25T13:18:06.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facfd9aa700 1 -- start start 2024-04-25T13:18:06.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facfd9aa700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf8102ac0 0x7facf8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:06.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facfd9aa700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf8103480 0x7facf8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:06.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facfd9aa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf8108a70 0x7facf819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:06.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facfd9aa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7facf8112d00 con 0x7facf8108a70 2024-04-25T13:18:06.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facfd9aa700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7facf8112b80 con 0x7facf8103480 2024-04-25T13:18:06.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facfd9aa700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7facf8112e80 con 0x7facf8102ac0 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facf77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf8108a70 0x7facf819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facf77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf8108a70 0x7facf819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49490/0 (socket says 172.21.15.31:49490) 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facf77fe700 1 -- 172.21.15.31:0/63484901 learned_addr learned my addr 172.21.15.31:0/63484901 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facf67fc700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf8103480 0x7facf8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facf6ffd700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf8102ac0 0x7facf8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.130+0000 7facf77fe700 1 -- 172.21.15.31:0/63484901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf8102ac0 msgr2=0x7facf8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.131+0000 7facf77fe700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf8102ac0 0x7facf8199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.131+0000 7facf77fe700 1 -- 172.21.15.31:0/63484901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf8103480 msgr2=0x7facf8199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.131+0000 7facf77fe700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf8103480 0x7facf8199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.131+0000 7facf77fe700 1 -- 172.21.15.31:0/63484901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf819e340 con 0x7facf8108a70 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.131+0000 7facf67fc700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7facf8103480 0x7facf8199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.131+0000 7facf77fe700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf8108a70 0x7facf819dc00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7facf000b970 tx=0x7facf000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:06.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.131+0000 7facfc9a8700 1 -- 172.21.15.31:0/63484901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf000d860 con 0x7facf8108a70 2024-04-25T13:18:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.131+0000 7facfd9aa700 1 -- 172.21.15.31:0/63484901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf819e630 con 0x7facf8108a70 2024-04-25T13:18:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.131+0000 7facfc9a8700 1 -- 172.21.15.31:0/63484901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf000e430 con 0x7facf8108a70 2024-04-25T13:18:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.131+0000 7facfc9a8700 1 -- 172.21.15.31:0/63484901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0017660 con 0x7facf8108a70 2024-04-25T13:18:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.131+0000 7facfd9aa700 1 -- 172.21.15.31:0/63484901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf81a5eb0 con 0x7facf8108a70 2024-04-25T13:18:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.132+0000 7facfd9aa700 1 -- 172.21.15.31:0/63484901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7facf80630d0 con 0x7facf8108a70 2024-04-25T13:18:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.133+0000 7facfc9a8700 1 -- 172.21.15.31:0/63484901 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7facf000da00 con 0x7facf8108a70 2024-04-25T13:18:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.134+0000 7facfc9a8700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7face00a3cd0 0x7face00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.134+0000 7facf6ffd700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7face00a3cd0 0x7face00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.134+0000 7facfc9a8700 1 -- 172.21.15.31:0/63484901 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7facf00d03d0 con 0x7facf8108a70 2024-04-25T13:18:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.134+0000 7facfc9a8700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670] conn(0x7face00ad4b0 0x7face00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.134+0000 7facfc9a8700 1 -- 172.21.15.31:0/63484901 --> [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facf0011ec0 con 0x7face00ad4b0 2024-04-25T13:18:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.134+0000 7facfc9a8700 1 -- 172.21.15.31:0/63484901 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7facf00d08b0 con 0x7facf8108a70 2024-04-25T13:18:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.134+0000 7facf67fc700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670] conn(0x7face00ad4b0 0x7face00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.134+0000 7facf6ffd700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7face00a3cd0 0x7face00a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7face800b5c0 tx=0x7face8005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.135+0000 7facf67fc700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670] conn(0x7face00ad4b0 0x7face00af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.135+0000 7facfc9a8700 1 -- 172.21.15.31:0/63484901 <== osd.0 v2:172.21.15.31:6802/2594221670 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facf0011ec0 con 0x7face00ad4b0 2024-04-25T13:18:06.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.152+0000 7facfd9aa700 1 -- 172.21.15.31:0/63484901 --> [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7facf80634d0 con 0x7face00ad4b0 2024-04-25T13:18:06.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.152+0000 7facfc9a8700 1 -- 172.21.15.31:0/63484901 <== osd.0 v2:172.21.15.31:6802/2594221670 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7facf80634d0 con 0x7face00ad4b0 2024-04-25T13:18:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.153+0000 7facfd9aa700 1 -- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670] conn(0x7face00ad4b0 msgr2=0x7face00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.153+0000 7facfd9aa700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670] conn(0x7face00ad4b0 0x7face00af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.153+0000 7facfd9aa700 1 -- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7face00a3cd0 msgr2=0x7face00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.153+0000 7facfd9aa700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7face00a3cd0 0x7face00a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7face800b5c0 tx=0x7face8005960 comp rx=0 tx=0).stop 2024-04-25T13:18:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.153+0000 7facfd9aa700 1 -- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf8108a70 msgr2=0x7facf819dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.153+0000 7facfd9aa700 1 --2- 172.21.15.31:0/63484901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facf8108a70 0x7facf819dc00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7facf000b970 tx=0x7facf000bc80 comp rx=0 tx=0).stop 2024-04-25T13:18:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.153+0000 7facf77fe700 1 -- 172.21.15.31:0/63484901 reap_dead start 2024-04-25T13:18:06.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.153+0000 7facfd9aa700 1 -- 172.21.15.31:0/63484901 shutdown_connections 2024-04-25T13:18:06.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.153+0000 7facfd9aa700 1 -- 172.21.15.31:0/63484901 >> 172.21.15.31:0/63484901 conn(0x7facf8079bb0 msgr2=0x7facf8101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:06.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.153+0000 7facfd9aa700 1 -- 172.21.15.31:0/63484901 shutdown_connections 2024-04-25T13:18:06.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:06.153+0000 7facfd9aa700 1 -- 172.21.15.31:0/63484901 wait complete. 2024-04-25T13:18:06.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:06 smithi031 ceph-mon[29457]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:07.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:06 smithi066 ceph-mon[33137]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:07.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:06 smithi136 ceph-mon[33038]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:08 smithi066 ceph-mon[33137]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:18:09.210 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:08 smithi031 ceph-mon[29457]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:18:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:08 smithi136 ceph-mon[33038]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:18:09.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.089+0000 7ff5151f5700 1 -- 172.21.15.31:0/951133662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510107b20 msgr2=0x7ff510107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:09.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.089+0000 7ff5151f5700 1 --2- 172.21.15.31:0/951133662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510107b20 0x7ff510107f00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff500009990 tx=0x7ff500009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.089+0000 7ff5151f5700 1 -- 172.21.15.31:0/951133662 shutdown_connections 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.089+0000 7ff5151f5700 1 --2- 172.21.15.31:0/951133662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff510101dc0 0x7ff51010f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.089+0000 7ff5151f5700 1 --2- 172.21.15.31:0/951133662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510107b20 0x7ff510107f00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.090+0000 7ff5151f5700 1 --2- 172.21.15.31:0/951133662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff510101400 0x7ff510101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.090+0000 7ff5151f5700 1 -- 172.21.15.31:0/951133662 >> 172.21.15.31:0/951133662 conn(0x7ff510076c60 msgr2=0x7ff510077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.090+0000 7ff5151f5700 1 -- 172.21.15.31:0/951133662 shutdown_connections 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.090+0000 7ff5151f5700 1 -- 172.21.15.31:0/951133662 wait complete. 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.090+0000 7ff5151f5700 1 Processor -- start 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff5151f5700 1 -- start start 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff5151f5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff510101400 0x7ff510199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff5151f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510101dc0 0x7ff510199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff5151f5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff510107b20 0x7ff51019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:09.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff5151f5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff510112c50 con 0x7ff510101dc0 2024-04-25T13:18:09.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff5151f5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff510112ad0 con 0x7ff510101400 2024-04-25T13:18:09.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff5151f5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff510112dd0 con 0x7ff510107b20 2024-04-25T13:18:09.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff50e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510101dc0 0x7ff510199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:09.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff50e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510101dc0 0x7ff510199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49516/0 (socket says 172.21.15.31:49516) 2024-04-25T13:18:09.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff50e59c700 1 -- 172.21.15.31:0/2798563421 learned_addr learned my addr 172.21.15.31:0/2798563421 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:09.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff50f59e700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff510107b20 0x7ff51019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:09.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff50ed9d700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff510101400 0x7ff510199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:09.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff50e59c700 1 -- 172.21.15.31:0/2798563421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff510107b20 msgr2=0x7ff51019d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff50e59c700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff510107b20 0x7ff51019d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff50e59c700 1 -- 172.21.15.31:0/2798563421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff510101400 msgr2=0x7ff510199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff50e59c700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff510101400 0x7ff510199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff50e59c700 1 -- 172.21.15.31:0/2798563421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51019e050 con 0x7ff510101dc0 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.091+0000 7ff50e59c700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510101dc0 0x7ff510199580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c9b0 tx=0x7ff50400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.092+0000 7ff4f7fff700 1 -- 172.21.15.31:0/2798563421 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504007b00 con 0x7ff510101dc0 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.092+0000 7ff5151f5700 1 -- 172.21.15.31:0/2798563421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51019e340 con 0x7ff510101dc0 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.092+0000 7ff4f7fff700 1 -- 172.21.15.31:0/2798563421 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff504007ca0 con 0x7ff510101dc0 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.092+0000 7ff4f7fff700 1 -- 172.21.15.31:0/2798563421 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504018660 con 0x7ff510101dc0 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.092+0000 7ff5151f5700 1 -- 172.21.15.31:0/2798563421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5101a5bc0 con 0x7ff510101dc0 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.092+0000 7ff5151f5700 1 -- 172.21.15.31:0/2798563421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff5100631d0 con 0x7ff510101dc0 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.094+0000 7ff4f7fff700 1 -- 172.21.15.31:0/2798563421 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff50400f450 con 0x7ff510101dc0 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.095+0000 7ff4f7fff700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff4f80a3cd0 0x7ff4f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:09.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.095+0000 7ff50ed9d700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff4f80a3cd0 0x7ff4f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.095+0000 7ff4f7fff700 1 -- 172.21.15.31:0/2798563421 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7ff5040d0aa0 con 0x7ff510101dc0 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.095+0000 7ff4f7fff700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269] conn(0x7ff4f80ad4b0 0x7ff4f80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.095+0000 7ff4f7fff700 1 -- 172.21.15.31:0/2798563421 --> [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4f80aff80 con 0x7ff4f80ad4b0 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.095+0000 7ff4f7fff700 1 -- 172.21.15.31:0/2798563421 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7ff5040d0e50 con 0x7ff510101dc0 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.095+0000 7ff50ed9d700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff4f80a3cd0 0x7ff4f80a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff500006010 tx=0x7ff50001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.095+0000 7ff50f59e700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269] conn(0x7ff4f80ad4b0 0x7ff4f80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.096+0000 7ff50f59e700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269] conn(0x7ff4f80ad4b0 0x7ff4f80af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.097+0000 7ff4f7fff700 1 -- 172.21.15.31:0/2798563421 <== osd.2 v2:172.21.15.136:6800/3445625269 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4f80aff80 con 0x7ff4f80ad4b0 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.113+0000 7ff5151f5700 1 -- 172.21.15.31:0/2798563421 --> [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff510063600 con 0x7ff4f80ad4b0 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff4f7fff700 1 -- 172.21.15.31:0/2798563421 <== osd.2 v2:172.21.15.136:6800/3445625269 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff510063600 con 0x7ff4f80ad4b0 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff5151f5700 1 -- 172.21.15.31:0/2798563421 >> [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269] conn(0x7ff4f80ad4b0 msgr2=0x7ff4f80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff5151f5700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269] conn(0x7ff4f80ad4b0 0x7ff4f80af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff5151f5700 1 -- 172.21.15.31:0/2798563421 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff4f80a3cd0 msgr2=0x7ff4f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff5151f5700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff4f80a3cd0 0x7ff4f80a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff500006010 tx=0x7ff50001a040 comp rx=0 tx=0).stop 2024-04-25T13:18:09.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff5151f5700 1 -- 172.21.15.31:0/2798563421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510101dc0 msgr2=0x7ff510199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:09.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff5151f5700 1 --2- 172.21.15.31:0/2798563421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510101dc0 0x7ff510199580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c9b0 tx=0x7ff50400ccc0 comp rx=0 tx=0).stop 2024-04-25T13:18:09.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff50f59e700 1 -- 172.21.15.31:0/2798563421 reap_dead start 2024-04-25T13:18:09.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff5151f5700 1 -- 172.21.15.31:0/2798563421 shutdown_connections 2024-04-25T13:18:09.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff5151f5700 1 -- 172.21.15.31:0/2798563421 >> 172.21.15.31:0/2798563421 conn(0x7ff510076c60 msgr2=0x7ff5100ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:09.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff5151f5700 1 -- 172.21.15.31:0/2798563421 shutdown_connections 2024-04-25T13:18:09.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:08.114+0000 7ff5151f5700 1 -- 172.21.15.31:0/2798563421 wait complete. 2024-04-25T13:18:10.001 INFO:teuthology.orchestra.run.smithi031.stdout:77309411393 2024-04-25T13:18:10.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T13:18:10.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.177+0000 7fa6fff54700 1 -- 172.21.15.31:0/708868813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f81022f0 msgr2=0x7fa6f810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.177+0000 7fa6fff54700 1 --2- 172.21.15.31:0/708868813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f81022f0 0x7fa6f810e7e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000b0a0 tx=0x7fa6f000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.177+0000 7fa6fff54700 1 -- 172.21.15.31:0/708868813 shutdown_connections 2024-04-25T13:18:10.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.177+0000 7fa6fff54700 1 --2- 172.21.15.31:0/708868813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f81022f0 0x7fa6f810e7e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.177+0000 7fa6fff54700 1 --2- 172.21.15.31:0/708868813 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f8107980 0x7fa6f8107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.177+0000 7fa6fff54700 1 --2- 172.21.15.31:0/708868813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f8101930 0x7fa6f8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.177+0000 7fa6fff54700 1 -- 172.21.15.31:0/708868813 >> 172.21.15.31:0/708868813 conn(0x7fa6f80788b0 msgr2=0x7fa6f8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:10.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.177+0000 7fa6fff54700 1 -- 172.21.15.31:0/708868813 shutdown_connections 2024-04-25T13:18:10.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.177+0000 7fa6fff54700 1 -- 172.21.15.31:0/708868813 wait complete. 2024-04-25T13:18:10.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.177+0000 7fa6fff54700 1 Processor -- start 2024-04-25T13:18:10.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fff54700 1 -- start start 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fff54700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f8101930 0x7fa6f819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fff54700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f81022f0 0x7fa6f819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fff54700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f8107980 0x7fa6f819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fff54700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8111b20 con 0x7fa6f8101930 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fff54700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81119a0 con 0x7fa6f81022f0 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fff54700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8111ca0 con 0x7fa6f8107980 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fdcf0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f8101930 0x7fa6f819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fdcf0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f8101930 0x7fa6f819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49530/0 (socket says 172.21.15.31:49530) 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fdcf0700 1 -- 172.21.15.31:0/2132558988 learned_addr learned my addr 172.21.15.31:0/2132558988 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fd4ef700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f81022f0 0x7fa6f819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fe4f1700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f8107980 0x7fa6f819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fdcf0700 1 -- 172.21.15.31:0/2132558988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f8107980 msgr2=0x7fa6f819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fdcf0700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f8107980 0x7fa6f819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fdcf0700 1 -- 172.21.15.31:0/2132558988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f81022f0 msgr2=0x7fa6f819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fdcf0700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6f81022f0 0x7fa6f819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fdcf0700 1 -- 172.21.15.31:0/2132558988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f81a0340 con 0x7fa6f8101930 2024-04-25T13:18:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.178+0000 7fa6fdcf0700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f8101930 0x7fa6f819b330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400e9a0 tx=0x7fa6f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.179+0000 7fa6eeffd700 1 -- 172.21.15.31:0/2132558988 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400cbd0 con 0x7fa6f8101930 2024-04-25T13:18:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.179+0000 7fa6fff54700 1 -- 172.21.15.31:0/2132558988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f81a0630 con 0x7fa6f8101930 2024-04-25T13:18:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.179+0000 7fa6eeffd700 1 -- 172.21.15.31:0/2132558988 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f400cd70 con 0x7fa6f8101930 2024-04-25T13:18:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.179+0000 7fa6eeffd700 1 -- 172.21.15.31:0/2132558988 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4018680 con 0x7fa6f8101930 2024-04-25T13:18:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.179+0000 7fa6fff54700 1 -- 172.21.15.31:0/2132558988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f81a7eb0 con 0x7fa6f8101930 2024-04-25T13:18:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.179+0000 7fa6fff54700 1 -- 172.21.15.31:0/2132558988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa6f806b8f0 con 0x7fa6f8101930 2024-04-25T13:18:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.181+0000 7fa6eeffd700 1 -- 172.21.15.31:0/2132558988 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa6f4010430 con 0x7fa6f8101930 2024-04-25T13:18:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.182+0000 7fa6eeffd700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa6e40a3cd0 0x7fa6e40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.182+0000 7fa6eeffd700 1 -- 172.21.15.31:0/2132558988 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa6f40d10c0 con 0x7fa6f8101930 2024-04-25T13:18:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.182+0000 7fa6fd4ef700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa6e40a3cd0 0x7fa6e40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.182+0000 7fa6eeffd700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860] conn(0x7fa6e40ad4b0 0x7fa6e40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.182+0000 7fa6eeffd700 1 -- 172.21.15.31:0/2132558988 --> [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa6e40aff80 con 0x7fa6e40ad4b0 2024-04-25T13:18:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.182+0000 7fa6eeffd700 1 -- 172.21.15.31:0/2132558988 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fa6f40d15a0 con 0x7fa6f8101930 2024-04-25T13:18:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.182+0000 7fa6fd4ef700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa6e40a3cd0 0x7fa6e40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8101860 tx=0x7fa6e800a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.182+0000 7fa6fe4f1700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860] conn(0x7fa6e40ad4b0 0x7fa6e40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.183+0000 7fa6fe4f1700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860] conn(0x7fa6e40ad4b0 0x7fa6e40af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.184+0000 7fa6eeffd700 1 -- 172.21.15.31:0/2132558988 <== osd.1 v2:172.21.15.66:6800/786007860 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa6e40aff80 con 0x7fa6e40ad4b0 2024-04-25T13:18:10.202 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:10.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.203+0000 7fa6fff54700 1 -- 172.21.15.31:0/2132558988 --> [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa6f8103050 con 0x7fa6e40ad4b0 2024-04-25T13:18:10.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.203+0000 7fa6eeffd700 1 -- 172.21.15.31:0/2132558988 <== osd.1 v2:172.21.15.66:6800/786007860 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa6f8103050 con 0x7fa6e40ad4b0 2024-04-25T13:18:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.204+0000 7fa6fff54700 1 -- 172.21.15.31:0/2132558988 >> [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860] conn(0x7fa6e40ad4b0 msgr2=0x7fa6e40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.204+0000 7fa6fff54700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860] conn(0x7fa6e40ad4b0 0x7fa6e40af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.204+0000 7fa6fff54700 1 -- 172.21.15.31:0/2132558988 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa6e40a3cd0 msgr2=0x7fa6e40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.204+0000 7fa6fff54700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa6e40a3cd0 0x7fa6e40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8101860 tx=0x7fa6e800a300 comp rx=0 tx=0).stop 2024-04-25T13:18:10.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.204+0000 7fa6fff54700 1 -- 172.21.15.31:0/2132558988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f8101930 msgr2=0x7fa6f819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.204+0000 7fa6fff54700 1 --2- 172.21.15.31:0/2132558988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6f8101930 0x7fa6f819b330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400e9a0 tx=0x7fa6f400ecb0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.204+0000 7fa6fe4f1700 1 -- 172.21.15.31:0/2132558988 reap_dead start 2024-04-25T13:18:10.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.204+0000 7fa6fff54700 1 -- 172.21.15.31:0/2132558988 shutdown_connections 2024-04-25T13:18:10.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.204+0000 7fa6fff54700 1 -- 172.21.15.31:0/2132558988 >> 172.21.15.31:0/2132558988 conn(0x7fa6f80788b0 msgr2=0x7fa6f806af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:10.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.204+0000 7fa6fff54700 1 -- 172.21.15.31:0/2132558988 shutdown_connections 2024-04-25T13:18:10.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.204+0000 7fa6fff54700 1 -- 172.21.15.31:0/2132558988 wait complete. 2024-04-25T13:18:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.635+0000 7f1708d3b700 1 -- 172.21.15.31:0/3868963487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17041022f0 msgr2=0x7f170410e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.635+0000 7f1708d3b700 1 --2- 172.21.15.31:0/3868963487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17041022f0 0x7f170410e7e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00b0a0 tx=0x7f16fc00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.635+0000 7f1708d3b700 1 -- 172.21.15.31:0/3868963487 shutdown_connections 2024-04-25T13:18:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.635+0000 7f1708d3b700 1 --2- 172.21.15.31:0/3868963487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1704107980 0x7f1704107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.635+0000 7f1708d3b700 1 --2- 172.21.15.31:0/3868963487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17041022f0 0x7f170410e7e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.635+0000 7f1708d3b700 1 --2- 172.21.15.31:0/3868963487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1704101930 0x7f1704101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.635+0000 7f1708d3b700 1 -- 172.21.15.31:0/3868963487 >> 172.21.15.31:0/3868963487 conn(0x7f17040788b0 msgr2=0x7f1704078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:10.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.635+0000 7f1708d3b700 1 -- 172.21.15.31:0/3868963487 shutdown_connections 2024-04-25T13:18:10.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.635+0000 7f1708d3b700 1 -- 172.21.15.31:0/3868963487 wait complete. 2024-04-25T13:18:10.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1708d3b700 1 Processor -- start 2024-04-25T13:18:10.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1708d3b700 1 -- start start 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1708d3b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1704101930 0x7f1704199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1708d3b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17041022f0 0x7f1704199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1708d3b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1704107980 0x7f170419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1708d3b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1704111af0 con 0x7f1704101930 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1708d3b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1704111970 con 0x7f17041022f0 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1708d3b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1704111c70 con 0x7f1704107980 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f170259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1704101930 0x7f1704199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f170259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1704101930 0x7f1704199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49560/0 (socket says 172.21.15.31:49560) 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f170259c700 1 -- 172.21.15.31:0/2433169759 learned_addr learned my addr 172.21.15.31:0/2433169759 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1702d9d700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1704107980 0x7f170419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1701d9b700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17041022f0 0x7f1704199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f170259c700 1 -- 172.21.15.31:0/2433169759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1704107980 msgr2=0x7f170419d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f170259c700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1704107980 0x7f170419d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f170259c700 1 -- 172.21.15.31:0/2433169759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17041022f0 msgr2=0x7f1704199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f170259c700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17041022f0 0x7f1704199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f170259c700 1 -- 172.21.15.31:0/2433169759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170419e050 con 0x7f1704101930 2024-04-25T13:18:10.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1701d9b700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17041022f0 0x7f1704199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:18:10.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.636+0000 7f1702d9d700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1704107980 0x7f170419d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:18:10.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.637+0000 7f170259c700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1704101930 0x7f1704199040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f16f400e9c0 tx=0x7f16f400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:10.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.637+0000 7f16f37fe700 1 -- 172.21.15.31:0/2433169759 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400cbd0 con 0x7f1704101930 2024-04-25T13:18:10.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.637+0000 7f16f37fe700 1 -- 172.21.15.31:0/2433169759 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f400cd70 con 0x7f1704101930 2024-04-25T13:18:10.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.637+0000 7f1708d3b700 1 -- 172.21.15.31:0/2433169759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170419e340 con 0x7f1704101930 2024-04-25T13:18:10.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.637+0000 7f16f37fe700 1 -- 172.21.15.31:0/2433169759 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4005610 con 0x7f1704101930 2024-04-25T13:18:10.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.637+0000 7f1708d3b700 1 -- 172.21.15.31:0/2433169759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17041a5bc0 con 0x7f1704101930 2024-04-25T13:18:10.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.638+0000 7f1708d3b700 1 -- 172.21.15.31:0/2433169759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f17040631d0 con 0x7f1704101930 2024-04-25T13:18:10.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.639+0000 7f16f37fe700 1 -- 172.21.15.31:0/2433169759 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f16f4004750 con 0x7f1704101930 2024-04-25T13:18:10.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.640+0000 7f16f37fe700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f16ec0a3cd0 0x7f16ec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.640+0000 7f1701d9b700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f16ec0a3cd0 0x7f16ec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.640+0000 7f16f37fe700 1 -- 172.21.15.31:0/2433169759 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f16f40d0a70 con 0x7f1704101930 2024-04-25T13:18:10.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.640+0000 7f16f37fe700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636] conn(0x7f16ec0ad4b0 0x7f16ec0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.640+0000 7f16f37fe700 1 -- 172.21.15.31:0/2433169759 --> [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16ec0aff80 con 0x7f16ec0ad4b0 2024-04-25T13:18:10.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.640+0000 7f16f37fe700 1 -- 172.21.15.31:0/2433169759 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f16f40d0f50 con 0x7f1704101930 2024-04-25T13:18:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.640+0000 7f1701d9b700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f16ec0a3cd0 0x7f16ec0a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f16f8005fd0 tx=0x7f16f8005e30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.640+0000 7f1702d9d700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636] conn(0x7f16ec0ad4b0 0x7f16ec0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.641+0000 7f1702d9d700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636] conn(0x7f16ec0ad4b0 0x7f16ec0af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:10.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.642+0000 7f16f37fe700 1 -- 172.21.15.31:0/2433169759 <== osd.10 v2:172.21.15.66:6824/2563017636 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16ec0aff80 con 0x7f16ec0ad4b0 2024-04-25T13:18:10.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.658+0000 7f1708d3b700 1 -- 172.21.15.31:0/2433169759 --> [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1704063650 con 0x7f16ec0ad4b0 2024-04-25T13:18:10.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.658+0000 7f16f37fe700 1 -- 172.21.15.31:0/2433169759 <== osd.10 v2:172.21.15.66:6824/2563017636 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1704063650 con 0x7f16ec0ad4b0 2024-04-25T13:18:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.659+0000 7f1708d3b700 1 -- 172.21.15.31:0/2433169759 >> [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636] conn(0x7f16ec0ad4b0 msgr2=0x7f16ec0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.659+0000 7f1708d3b700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636] conn(0x7f16ec0ad4b0 0x7f16ec0af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.659+0000 7f1708d3b700 1 -- 172.21.15.31:0/2433169759 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f16ec0a3cd0 msgr2=0x7f16ec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.659+0000 7f1708d3b700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f16ec0a3cd0 0x7f16ec0a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f16f8005fd0 tx=0x7f16f8005e30 comp rx=0 tx=0).stop 2024-04-25T13:18:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.659+0000 7f1708d3b700 1 -- 172.21.15.31:0/2433169759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1704101930 msgr2=0x7f1704199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.659+0000 7f1708d3b700 1 --2- 172.21.15.31:0/2433169759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1704101930 0x7f1704199040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f16f400e9c0 tx=0x7f16f400ecd0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.659+0000 7f1702d9d700 1 -- 172.21.15.31:0/2433169759 reap_dead start 2024-04-25T13:18:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.659+0000 7f1708d3b700 1 -- 172.21.15.31:0/2433169759 shutdown_connections 2024-04-25T13:18:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.659+0000 7f1708d3b700 1 -- 172.21.15.31:0/2433169759 >> 172.21.15.31:0/2433169759 conn(0x7f17040788b0 msgr2=0x7f170406b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.659+0000 7f1708d3b700 1 -- 172.21.15.31:0/2433169759 shutdown_connections 2024-04-25T13:18:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:10.659+0000 7f1708d3b700 1 -- 172.21.15.31:0/2433169759 wait complete. 2024-04-25T13:18:11.099 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:10 smithi031 ceph-mon[29457]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:10 smithi066 ceph-mon[33137]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:10 smithi136 ceph-mon[33038]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:12.551 INFO:teuthology.orchestra.run.smithi031.stdout:55834574917 2024-04-25T13:18:12.552 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T13:18:13.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:12 smithi136 ceph-mon[33038]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:13.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:12 smithi031 ceph-mon[29457]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:13.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:12 smithi066 ceph-mon[33137]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:13.509 INFO:teuthology.orchestra.run.smithi031.stdout:322122547223 2024-04-25T13:18:13.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T13:18:14.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.108+0000 7f595db57700 1 -- 172.21.15.31:0/1647241612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595806bbb0 msgr2=0x7f595810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.108+0000 7f595db57700 1 --2- 172.21.15.31:0/1647241612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595806bbb0 0x7f595810f930 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f595000b0a0 tx=0x7f595000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.108+0000 7f595db57700 1 -- 172.21.15.31:0/1647241612 shutdown_connections 2024-04-25T13:18:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.108+0000 7f595db57700 1 --2- 172.21.15.31:0/1647241612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f595806bbb0 0x7f595810f930 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.108+0000 7f595db57700 1 --2- 172.21.15.31:0/1647241612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5958108a60 0x7f5958108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.108+0000 7f595db57700 1 --2- 172.21.15.31:0/1647241612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f595806b1f0 0x7f595806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.108+0000 7f595db57700 1 -- 172.21.15.31:0/1647241612 >> 172.21.15.31:0/1647241612 conn(0x7f59580789e0 msgr2=0x7f5958078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.108+0000 7f595db57700 1 -- 172.21.15.31:0/1647241612 shutdown_connections 2024-04-25T13:18:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.108+0000 7f595db57700 1 -- 172.21.15.31:0/1647241612 wait complete. 2024-04-25T13:18:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f595db57700 1 Processor -- start 2024-04-25T13:18:14.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f595db57700 1 -- start start 2024-04-25T13:18:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f595db57700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f595806b1f0 0x7f5958199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f595db57700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595806bbb0 0x7f59581995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f595db57700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958108a60 0x7f595819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f595db57700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5958112c60 con 0x7f5958108a60 2024-04-25T13:18:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f595db57700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5958112ae0 con 0x7f595806bbb0 2024-04-25T13:18:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f595db57700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5958112de0 con 0x7f595806b1f0 2024-04-25T13:18:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f5957fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958108a60 0x7f595819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f5957fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958108a60 0x7f595819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45680/0 (socket says 172.21.15.31:45680) 2024-04-25T13:18:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f5957fff700 1 -- 172.21.15.31:0/1458331793 learned_addr learned my addr 172.21.15.31:0/1458331793 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f59577fe700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f595806b1f0 0x7f5958199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:14.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f5956ffd700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595806bbb0 0x7f59581995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f5957fff700 1 -- 172.21.15.31:0/1458331793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f595806b1f0 msgr2=0x7f5958199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f5957fff700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f595806b1f0 0x7f5958199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f5957fff700 1 -- 172.21.15.31:0/1458331793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595806bbb0 msgr2=0x7f59581995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f5957fff700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f595806bbb0 0x7f59581995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.109+0000 7f5957fff700 1 -- 172.21.15.31:0/1458331793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595819e0a0 con 0x7f5958108a60 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.110+0000 7f5957fff700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958108a60 0x7f595819d960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f595000b830 tx=0x7f5950004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.110+0000 7f5954ff9700 1 -- 172.21.15.31:0/1458331793 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950021070 con 0x7f5958108a60 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.110+0000 7f5954ff9700 1 -- 172.21.15.31:0/1458331793 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5950007e30 con 0x7f5958108a60 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.110+0000 7f595db57700 1 -- 172.21.15.31:0/1458331793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595819e330 con 0x7f5958108a60 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.110+0000 7f5954ff9700 1 -- 172.21.15.31:0/1458331793 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950004700 con 0x7f5958108a60 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.110+0000 7f595db57700 1 -- 172.21.15.31:0/1458331793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59581a5c10 con 0x7f5958108a60 2024-04-25T13:18:14.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.111+0000 7f595db57700 1 -- 172.21.15.31:0/1458331793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5958063190 con 0x7f5958108a60 2024-04-25T13:18:14.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.112+0000 7f5954ff9700 1 -- 172.21.15.31:0/1458331793 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5950003d70 con 0x7f5958108a60 2024-04-25T13:18:14.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.113+0000 7f5954ff9700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f59400a3cd0 0x7f59400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:14.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.113+0000 7f59577fe700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f59400a3cd0 0x7f59400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:14.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.113+0000 7f5954ff9700 1 -- 172.21.15.31:0/1458331793 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f595000f020 con 0x7f5958108a60 2024-04-25T13:18:14.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.113+0000 7f5954ff9700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941] conn(0x7f59400ad4b0 0x7f59400af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:14.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.113+0000 7f5954ff9700 1 -- 172.21.15.31:0/1458331793 --> [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f595001de50 con 0x7f59400ad4b0 2024-04-25T13:18:14.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.113+0000 7f5954ff9700 1 -- 172.21.15.31:0/1458331793 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f59500d1f70 con 0x7f5958108a60 2024-04-25T13:18:14.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.113+0000 7f59577fe700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f59400a3cd0 0x7f59400a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5948005950 tx=0x7f594800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:14.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.113+0000 7f5956ffd700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941] conn(0x7f59400ad4b0 0x7f59400af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:14.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.114+0000 7f5956ffd700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941] conn(0x7f59400ad4b0 0x7f59400af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:14.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.115+0000 7f5954ff9700 1 -- 172.21.15.31:0/1458331793 <== osd.11 v2:172.21.15.136:6824/2269044941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f595001de50 con 0x7f59400ad4b0 2024-04-25T13:18:14.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.131+0000 7f595db57700 1 -- 172.21.15.31:0/1458331793 --> [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5958063610 con 0x7f59400ad4b0 2024-04-25T13:18:14.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f5954ff9700 1 -- 172.21.15.31:0/1458331793 <== osd.11 v2:172.21.15.136:6824/2269044941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5958063610 con 0x7f59400ad4b0 2024-04-25T13:18:14.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f595db57700 1 -- 172.21.15.31:0/1458331793 >> [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941] conn(0x7f59400ad4b0 msgr2=0x7f59400af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:14.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f595db57700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941] conn(0x7f59400ad4b0 0x7f59400af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:14.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f595db57700 1 -- 172.21.15.31:0/1458331793 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f59400a3cd0 msgr2=0x7f59400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:14.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f595db57700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f59400a3cd0 0x7f59400a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5948005950 tx=0x7f594800b410 comp rx=0 tx=0).stop 2024-04-25T13:18:14.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f595db57700 1 -- 172.21.15.31:0/1458331793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958108a60 msgr2=0x7f595819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:14.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f595db57700 1 --2- 172.21.15.31:0/1458331793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958108a60 0x7f595819d960 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f595000b830 tx=0x7f5950004ce0 comp rx=0 tx=0).stop 2024-04-25T13:18:14.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f5957fff700 1 -- 172.21.15.31:0/1458331793 reap_dead start 2024-04-25T13:18:14.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f595db57700 1 -- 172.21.15.31:0/1458331793 shutdown_connections 2024-04-25T13:18:14.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f595db57700 1 -- 172.21.15.31:0/1458331793 >> 172.21.15.31:0/1458331793 conn(0x7f59580789e0 msgr2=0x7f59580ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:14.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f595db57700 1 -- 172.21.15.31:0/1458331793 shutdown_connections 2024-04-25T13:18:14.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:14.132+0000 7f595db57700 1 -- 172.21.15.31:0/1458331793 wait complete. 2024-04-25T13:18:14.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:14 smithi031 ceph-mon[29457]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:15.045 INFO:teuthology.orchestra.run.smithi031.stdout:34359738443 2024-04-25T13:18:15.045 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T13:18:15.209 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:14 smithi066 ceph-mon[33137]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:14 smithi136 ceph-mon[33038]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:18:17.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:16 smithi136 ceph-mon[33038]: pgmap v310: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:17.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:16 smithi031 ceph-mon[29457]: pgmap v310: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:17.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:16 smithi066 ceph-mon[33137]: pgmap v310: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:17.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.566+0000 7fa03bd35700 1 -- 172.21.15.31:0/4036001540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa03406bbb0 msgr2=0x7fa03410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:17.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.566+0000 7fa03bd35700 1 --2- 172.21.15.31:0/4036001540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa03406bbb0 0x7fa03410f930 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009990 tx=0x7fa02c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:17.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.566+0000 7fa03bd35700 1 -- 172.21.15.31:0/4036001540 shutdown_connections 2024-04-25T13:18:17.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.566+0000 7fa03bd35700 1 --2- 172.21.15.31:0/4036001540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa034108a60 0x7fa034108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:17.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.566+0000 7fa03bd35700 1 --2- 172.21.15.31:0/4036001540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa03406bbb0 0x7fa03410f930 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:17.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.566+0000 7fa03bd35700 1 --2- 172.21.15.31:0/4036001540 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa03406b1f0 0x7fa03406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:17.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.566+0000 7fa03bd35700 1 -- 172.21.15.31:0/4036001540 >> 172.21.15.31:0/4036001540 conn(0x7fa0340789e0 msgr2=0x7fa034078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:17.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.566+0000 7fa03bd35700 1 -- 172.21.15.31:0/4036001540 shutdown_connections 2024-04-25T13:18:17.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.566+0000 7fa03bd35700 1 -- 172.21.15.31:0/4036001540 wait complete. 2024-04-25T13:18:17.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03bd35700 1 Processor -- start 2024-04-25T13:18:17.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03bd35700 1 -- start start 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03bd35700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa03406b1f0 0x7fa0341990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03bd35700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa03406bbb0 0x7fa034199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03bd35700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa034108a60 0x7fa03419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03bd35700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa034112ca0 con 0x7fa034108a60 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03bd35700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa034112b20 con 0x7fa03406b1f0 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03bd35700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa034112e20 con 0x7fa03406bbb0 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03a2d2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa034108a60 0x7fa03419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03a2d2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa034108a60 0x7fa03419d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45690/0 (socket says 172.21.15.31:45690) 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03a2d2700 1 -- 172.21.15.31:0/3698209586 learned_addr learned my addr 172.21.15.31:0/3698209586 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa039ad1700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa03406b1f0 0x7fa0341990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa0392d0700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa03406bbb0 0x7fa034199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03a2d2700 1 -- 172.21.15.31:0/3698209586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa03406bbb0 msgr2=0x7fa034199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:17.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03a2d2700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa03406bbb0 0x7fa034199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03a2d2700 1 -- 172.21.15.31:0/3698209586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa03406b1f0 msgr2=0x7fa0341990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03a2d2700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa03406b1f0 0x7fa0341990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.567+0000 7fa03a2d2700 1 -- 172.21.15.31:0/3698209586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03419e090 con 0x7fa034108a60 2024-04-25T13:18:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.568+0000 7fa03a2d2700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa034108a60 0x7fa03419d950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa02c005910 tx=0x7fa02c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.568+0000 7fa032ffd700 1 -- 172.21.15.31:0/3698209586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c027070 con 0x7fa034108a60 2024-04-25T13:18:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.568+0000 7fa032ffd700 1 -- 172.21.15.31:0/3698209586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa02c003fa0 con 0x7fa034108a60 2024-04-25T13:18:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.568+0000 7fa03bd35700 1 -- 172.21.15.31:0/3698209586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03419e320 con 0x7fa034108a60 2024-04-25T13:18:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.568+0000 7fa032ffd700 1 -- 172.21.15.31:0/3698209586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c022d70 con 0x7fa034108a60 2024-04-25T13:18:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.568+0000 7fa03bd35700 1 -- 172.21.15.31:0/3698209586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0341a5c00 con 0x7fa034108a60 2024-04-25T13:18:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.569+0000 7fa03bd35700 1 -- 172.21.15.31:0/3698209586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0340631d0 con 0x7fa034108a60 2024-04-25T13:18:17.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.570+0000 7fa032ffd700 1 -- 172.21.15.31:0/3698209586 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa02c02a070 con 0x7fa034108a60 2024-04-25T13:18:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.571+0000 7fa032ffd700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa01c0a3d50 0x7fa01c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.571+0000 7fa039ad1700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa01c0a3d50 0x7fa01c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.571+0000 7fa032ffd700 1 -- 172.21.15.31:0/3698209586 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fa02c0d1c00 con 0x7fa034108a60 2024-04-25T13:18:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.571+0000 7fa032ffd700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945] conn(0x7fa01c0ad530 0x7fa01c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.571+0000 7fa032ffd700 1 -- 172.21.15.31:0/3698209586 --> [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa02c0043d0 con 0x7fa01c0ad530 2024-04-25T13:18:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.571+0000 7fa032ffd700 1 -- 172.21.15.31:0/3698209586 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fa02c0d20e0 con 0x7fa034108a60 2024-04-25T13:18:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.571+0000 7fa039ad1700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa01c0a3d50 0x7fa01c0a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa02400a400 tx=0x7fa02400d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.571+0000 7fa0392d0700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945] conn(0x7fa01c0ad530 0x7fa01c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.572+0000 7fa0392d0700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945] conn(0x7fa01c0ad530 0x7fa01c0af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.572+0000 7fa032ffd700 1 -- 172.21.15.31:0/3698209586 <== osd.7 v2:172.21.15.66:6816/2705866945 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa02c0043d0 con 0x7fa01c0ad530 2024-04-25T13:18:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.589+0000 7fa03bd35700 1 -- 172.21.15.31:0/3698209586 --> [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0340635d0 con 0x7fa01c0ad530 2024-04-25T13:18:17.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa032ffd700 1 -- 172.21.15.31:0/3698209586 <== osd.7 v2:172.21.15.66:6816/2705866945 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa0340635d0 con 0x7fa01c0ad530 2024-04-25T13:18:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa03bd35700 1 -- 172.21.15.31:0/3698209586 >> [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945] conn(0x7fa01c0ad530 msgr2=0x7fa01c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa03bd35700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945] conn(0x7fa01c0ad530 0x7fa01c0af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa03bd35700 1 -- 172.21.15.31:0/3698209586 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa01c0a3d50 msgr2=0x7fa01c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa03bd35700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa01c0a3d50 0x7fa01c0a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa02400a400 tx=0x7fa02400d040 comp rx=0 tx=0).stop 2024-04-25T13:18:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa03bd35700 1 -- 172.21.15.31:0/3698209586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa034108a60 msgr2=0x7fa03419d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa03bd35700 1 --2- 172.21.15.31:0/3698209586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa034108a60 0x7fa03419d950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa02c005910 tx=0x7fa02c0047b0 comp rx=0 tx=0).stop 2024-04-25T13:18:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa03a2d2700 1 -- 172.21.15.31:0/3698209586 reap_dead start 2024-04-25T13:18:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa03bd35700 1 -- 172.21.15.31:0/3698209586 shutdown_connections 2024-04-25T13:18:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa03bd35700 1 -- 172.21.15.31:0/3698209586 >> 172.21.15.31:0/3698209586 conn(0x7fa0340789e0 msgr2=0x7fa0340ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa03bd35700 1 -- 172.21.15.31:0/3698209586 shutdown_connections 2024-04-25T13:18:17.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:17.590+0000 7fa03bd35700 1 -- 172.21.15.31:0/3698209586 wait complete. 2024-04-25T13:18:18.547 INFO:teuthology.orchestra.run.smithi031.stdout:356482285588 2024-04-25T13:18:18.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T13:18:19.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:18 smithi031 ceph-mon[29457]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:19.163 INFO:teuthology.orchestra.run.smithi031.stdout:227633266728 2024-04-25T13:18:19.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T13:18:19.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:18 smithi136 ceph-mon[33038]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:19.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:18 smithi066 ceph-mon[33137]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.660+0000 7f3c3fe65700 1 -- 172.21.15.31:0/2007657990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3806bbb0 msgr2=0x7f3c3810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.660+0000 7f3c3fe65700 1 --2- 172.21.15.31:0/2007657990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3806bbb0 0x7f3c3810f930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000b0a0 tx=0x7f3c3000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.660+0000 7f3c3fe65700 1 -- 172.21.15.31:0/2007657990 shutdown_connections 2024-04-25T13:18:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.660+0000 7f3c3fe65700 1 --2- 172.21.15.31:0/2007657990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3806bbb0 0x7f3c3810f930 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.660+0000 7f3c3fe65700 1 --2- 172.21.15.31:0/2007657990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c38108a60 0x7f3c38108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.660+0000 7f3c3fe65700 1 --2- 172.21.15.31:0/2007657990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c3806b1f0 0x7f3c3806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.660+0000 7f3c3fe65700 1 -- 172.21.15.31:0/2007657990 >> 172.21.15.31:0/2007657990 conn(0x7f3c380789e0 msgr2=0x7f3c38078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.660+0000 7f3c3fe65700 1 -- 172.21.15.31:0/2007657990 shutdown_connections 2024-04-25T13:18:19.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.660+0000 7f3c3fe65700 1 -- 172.21.15.31:0/2007657990 wait complete. 2024-04-25T13:18:19.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.661+0000 7f3c3fe65700 1 Processor -- start 2024-04-25T13:18:19.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.661+0000 7f3c3fe65700 1 -- start start 2024-04-25T13:18:19.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.661+0000 7f3c3fe65700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3806b1f0 0x7f3c381019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:19.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.661+0000 7f3c3fe65700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3806bbb0 0x7f3c38101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:19.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.661+0000 7f3c3fe65700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c38108a60 0x7f3c38102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:19.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.661+0000 7f3c3fe65700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3c38112c40 con 0x7f3c3806b1f0 2024-04-25T13:18:19.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.661+0000 7f3c3fe65700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3c38112ac0 con 0x7f3c3806bbb0 2024-04-25T13:18:19.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.661+0000 7f3c3fe65700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3c38112dc0 con 0x7f3c38108a60 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.661+0000 7f3c3dc01700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3806b1f0 0x7f3c381019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.661+0000 7f3c3dc01700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3806b1f0 0x7f3c381019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45704/0 (socket says 172.21.15.31:45704) 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.661+0000 7f3c3dc01700 1 -- 172.21.15.31:0/123057180 learned_addr learned my addr 172.21.15.31:0/123057180 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c3d400700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3806bbb0 0x7f3c38101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c3e402700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c38108a60 0x7f3c38102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c3dc01700 1 -- 172.21.15.31:0/123057180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c38108a60 msgr2=0x7f3c38102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c3dc01700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c38108a60 0x7f3c38102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c3dc01700 1 -- 172.21.15.31:0/123057180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3806bbb0 msgr2=0x7f3c38101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c3dc01700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3806bbb0 0x7f3c38101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c3dc01700 1 -- 172.21.15.31:0/123057180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c381a7620 con 0x7f3c3806b1f0 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c3dc01700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3806b1f0 0x7f3c381019b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400d950 tx=0x7f3c3400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c2effd700 1 -- 172.21.15.31:0/123057180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34016070 con 0x7f3c3806b1f0 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c3fe65700 1 -- 172.21.15.31:0/123057180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c381a7910 con 0x7f3c3806b1f0 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c2effd700 1 -- 172.21.15.31:0/123057180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c3400fd40 con 0x7f3c3806b1f0 2024-04-25T13:18:19.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c2effd700 1 -- 172.21.15.31:0/123057180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3400e3b0 con 0x7f3c3806b1f0 2024-04-25T13:18:19.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.662+0000 7f3c3fe65700 1 -- 172.21.15.31:0/123057180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c381a7e50 con 0x7f3c3806b1f0 2024-04-25T13:18:19.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.663+0000 7f3c3fe65700 1 -- 172.21.15.31:0/123057180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c38063110 con 0x7f3c3806b1f0 2024-04-25T13:18:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.664+0000 7f3c2effd700 1 -- 172.21.15.31:0/123057180 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3c34019070 con 0x7f3c3806b1f0 2024-04-25T13:18:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.665+0000 7f3c2effd700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3c240a3cd0 0x7f3c240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.665+0000 7f3c3d400700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3c240a3cd0 0x7f3c240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.665+0000 7f3c2effd700 1 -- 172.21.15.31:0/123057180 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f3c340d0340 con 0x7f3c3806b1f0 2024-04-25T13:18:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.665+0000 7f3c2effd700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958] conn(0x7f3c240ad4b0 0x7f3c240af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.665+0000 7f3c2effd700 1 -- 172.21.15.31:0/123057180 --> [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c34014e80 con 0x7f3c240ad4b0 2024-04-25T13:18:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.665+0000 7f3c2effd700 1 -- 172.21.15.31:0/123057180 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f3c340d0820 con 0x7f3c3806b1f0 2024-04-25T13:18:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.665+0000 7f3c3e402700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958] conn(0x7f3c240ad4b0 0x7f3c240af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.665+0000 7f3c3d400700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3c240a3cd0 0x7f3c240a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c38078ee0 tx=0x7f3c2800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:19.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.666+0000 7f3c3e402700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958] conn(0x7f3c240ad4b0 0x7f3c240af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:19.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.666+0000 7f3c2effd700 1 -- 172.21.15.31:0/123057180 <== osd.3 v2:172.21.15.31:6810/1987517958 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c34014e80 con 0x7f3c240ad4b0 2024-04-25T13:18:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.683+0000 7f3c3fe65700 1 -- 172.21.15.31:0/123057180 --> [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3c38063510 con 0x7f3c240ad4b0 2024-04-25T13:18:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.683+0000 7f3c2effd700 1 -- 172.21.15.31:0/123057180 <== osd.3 v2:172.21.15.31:6810/1987517958 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3c38063510 con 0x7f3c240ad4b0 2024-04-25T13:18:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.683+0000 7f3c3fe65700 1 -- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958] conn(0x7f3c240ad4b0 msgr2=0x7f3c240af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.683+0000 7f3c3fe65700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958] conn(0x7f3c240ad4b0 0x7f3c240af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.683+0000 7f3c3fe65700 1 -- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3c240a3cd0 msgr2=0x7f3c240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.683+0000 7f3c3fe65700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3c240a3cd0 0x7f3c240a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c38078ee0 tx=0x7f3c2800b410 comp rx=0 tx=0).stop 2024-04-25T13:18:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.683+0000 7f3c3fe65700 1 -- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3806b1f0 msgr2=0x7f3c381019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.683+0000 7f3c3fe65700 1 --2- 172.21.15.31:0/123057180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c3806b1f0 0x7f3c381019b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400d950 tx=0x7f3c3400dc60 comp rx=0 tx=0).stop 2024-04-25T13:18:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.683+0000 7f3c3e402700 1 -- 172.21.15.31:0/123057180 reap_dead start 2024-04-25T13:18:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.683+0000 7f3c3fe65700 1 -- 172.21.15.31:0/123057180 shutdown_connections 2024-04-25T13:18:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.683+0000 7f3c3fe65700 1 -- 172.21.15.31:0/123057180 >> 172.21.15.31:0/123057180 conn(0x7f3c380789e0 msgr2=0x7f3c380ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.684+0000 7f3c3fe65700 1 -- 172.21.15.31:0/123057180 shutdown_connections 2024-04-25T13:18:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:19.684+0000 7f3c3fe65700 1 -- 172.21.15.31:0/123057180 wait complete. 2024-04-25T13:18:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:115964117055 2024-04-25T13:18:20.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T13:18:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.853+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1217062748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc108a50 msgr2=0x7f4bcc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.853+0000 7f4bd1cc7700 1 --2- 172.21.15.31:0/1217062748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc108a50 0x7f4bcc108e30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009990 tx=0x7f4bbc009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.853+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1217062748 shutdown_connections 2024-04-25T13:18:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.853+0000 7f4bd1cc7700 1 --2- 172.21.15.31:0/1217062748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bcc103450 0x7f4bcc10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.853+0000 7f4bd1cc7700 1 --2- 172.21.15.31:0/1217062748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc108a50 0x7f4bcc108e30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.853+0000 7f4bd1cc7700 1 --2- 172.21.15.31:0/1217062748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bcc102a90 0x7f4bcc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.853+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1217062748 >> 172.21.15.31:0/1217062748 conn(0x7f4bcc079ba0 msgr2=0x7f4bcc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.853+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1217062748 shutdown_connections 2024-04-25T13:18:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.853+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1217062748 wait complete. 2024-04-25T13:18:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bd1cc7700 1 Processor -- start 2024-04-25T13:18:20.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bd1cc7700 1 -- start start 2024-04-25T13:18:20.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bd1cc7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bcc102a90 0x7f4bcc199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:20.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bd1cc7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103450 0x7f4bcc199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:20.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bd1cc7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bcc108a50 0x7f4bcc19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:20.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bd1cc7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc112c00 con 0x7f4bcc103450 2024-04-25T13:18:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bd1cc7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc112a80 con 0x7f4bcc108a50 2024-04-25T13:18:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bd1cc7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc112d80 con 0x7f4bcc102a90 2024-04-25T13:18:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bcaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103450 0x7f4bcc199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bcaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103450 0x7f4bcc199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45722/0 (socket says 172.21.15.31:45722) 2024-04-25T13:18:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bcaffd700 1 -- 172.21.15.31:0/1170110751 learned_addr learned my addr 172.21.15.31:0/1170110751 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bcb7fe700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bcc102a90 0x7f4bcc199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bcbfff700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bcc108a50 0x7f4bcc19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bcaffd700 1 -- 172.21.15.31:0/1170110751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bcc102a90 msgr2=0x7f4bcc199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bcaffd700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bcc102a90 0x7f4bcc199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bcaffd700 1 -- 172.21.15.31:0/1170110751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bcc108a50 msgr2=0x7f4bcc19d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bcaffd700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4bcc108a50 0x7f4bcc19d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:20.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.854+0000 7f4bcaffd700 1 -- 172.21.15.31:0/1170110751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc19e060 con 0x7f4bcc103450 2024-04-25T13:18:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.855+0000 7f4bcaffd700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103450 0x7f4bcc199590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c9b0 tx=0x7f4bc000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.855+0000 7f4bc8ff9700 1 -- 172.21.15.31:0/1170110751 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0007b00 con 0x7f4bcc103450 2024-04-25T13:18:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.855+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1170110751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc19e350 con 0x7f4bcc103450 2024-04-25T13:18:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.855+0000 7f4bc8ff9700 1 -- 172.21.15.31:0/1170110751 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bc000ce50 con 0x7f4bcc103450 2024-04-25T13:18:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.855+0000 7f4bc8ff9700 1 -- 172.21.15.31:0/1170110751 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0018660 con 0x7f4bcc103450 2024-04-25T13:18:20.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.855+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1170110751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc1a5bd0 con 0x7f4bcc103450 2024-04-25T13:18:20.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.856+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1170110751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4bcc0631d0 con 0x7f4bcc103450 2024-04-25T13:18:20.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.857+0000 7f4bc8ff9700 1 -- 172.21.15.31:0/1170110751 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4bc000f450 con 0x7f4bcc103450 2024-04-25T13:18:20.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.858+0000 7f4bc8ff9700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f4bb40a3cd0 0x7f4bb40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:20.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.858+0000 7f4bcb7fe700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f4bb40a3cd0 0x7f4bb40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:20.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.858+0000 7f4bc8ff9700 1 -- 172.21.15.31:0/1170110751 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f4bc00d0740 con 0x7f4bcc103450 2024-04-25T13:18:20.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.858+0000 7f4bc8ff9700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340] conn(0x7f4bb40ad4b0 0x7f4bb40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:20.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.858+0000 7f4bc8ff9700 1 -- 172.21.15.31:0/1170110751 --> [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4bb40aff80 con 0x7f4bb40ad4b0 2024-04-25T13:18:20.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.858+0000 7f4bc8ff9700 1 -- 172.21.15.31:0/1170110751 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f4bc00d0c20 con 0x7f4bcc103450 2024-04-25T13:18:20.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.858+0000 7f4bcbfff700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340] conn(0x7f4bb40ad4b0 0x7f4bb40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:20.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.858+0000 7f4bcb7fe700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f4bb40a3cd0 0x7f4bb40a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc006010 tx=0x7f4bbc00b560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.858+0000 7f4bcbfff700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340] conn(0x7f4bb40ad4b0 0x7f4bb40af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:20.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.859+0000 7f4bc8ff9700 1 -- 172.21.15.31:0/1170110751 <== osd.9 v2:172.21.15.31:6826/301987340 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bb40aff80 con 0x7f4bb40ad4b0 2024-04-25T13:18:20.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.875+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1170110751 --> [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4bcc0635d0 con 0x7f4bb40ad4b0 2024-04-25T13:18:20.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.876+0000 7f4bc8ff9700 1 -- 172.21.15.31:0/1170110751 <== osd.9 v2:172.21.15.31:6826/301987340 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4bcc0635d0 con 0x7f4bb40ad4b0 2024-04-25T13:18:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.876+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340] conn(0x7f4bb40ad4b0 msgr2=0x7f4bb40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.876+0000 7f4bd1cc7700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340] conn(0x7f4bb40ad4b0 0x7f4bb40af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.876+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f4bb40a3cd0 msgr2=0x7f4bb40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.876+0000 7f4bd1cc7700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f4bb40a3cd0 0x7f4bb40a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc006010 tx=0x7f4bbc00b560 comp rx=0 tx=0).stop 2024-04-25T13:18:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.876+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103450 msgr2=0x7f4bcc199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.876+0000 7f4bd1cc7700 1 --2- 172.21.15.31:0/1170110751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103450 0x7f4bcc199590 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c9b0 tx=0x7f4bc000ccc0 comp rx=0 tx=0).stop 2024-04-25T13:18:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.876+0000 7f4bcbfff700 1 -- 172.21.15.31:0/1170110751 reap_dead start 2024-04-25T13:18:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.876+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1170110751 shutdown_connections 2024-04-25T13:18:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.876+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1170110751 >> 172.21.15.31:0/1170110751 conn(0x7f4bcc079ba0 msgr2=0x7f4bcc101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.877+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1170110751 shutdown_connections 2024-04-25T13:18:20.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:20.877+0000 7f4bd1cc7700 1 -- 172.21.15.31:0/1170110751 wait complete. 2024-04-25T13:18:21.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:20 smithi136 ceph-mon[33038]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:21.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:20 smithi031 ceph-mon[29457]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:20 smithi066 ceph-mon[33137]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.019+0000 7f6d270c7700 1 -- 172.21.15.31:0/2756120324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20102ac0 msgr2=0x7f6d20102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.019+0000 7f6d270c7700 1 --2- 172.21.15.31:0/2756120324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20102ac0 0x7f6d20102f40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009990 tx=0x7f6d0c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.019+0000 7f6d270c7700 1 -- 172.21.15.31:0/2756120324 shutdown_connections 2024-04-25T13:18:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.019+0000 7f6d270c7700 1 --2- 172.21.15.31:0/2756120324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d20108a70 0x7f6d20108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.019+0000 7f6d270c7700 1 --2- 172.21.15.31:0/2756120324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d20103480 0x7f6d2010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.019+0000 7f6d270c7700 1 --2- 172.21.15.31:0/2756120324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20102ac0 0x7f6d20102f40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.019+0000 7f6d270c7700 1 -- 172.21.15.31:0/2756120324 >> 172.21.15.31:0/2756120324 conn(0x7f6d20079bb0 msgr2=0x7f6d20079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.019+0000 7f6d270c7700 1 -- 172.21.15.31:0/2756120324 shutdown_connections 2024-04-25T13:18:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.019+0000 7f6d270c7700 1 -- 172.21.15.31:0/2756120324 wait complete. 2024-04-25T13:18:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.019+0000 7f6d270c7700 1 Processor -- start 2024-04-25T13:18:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d270c7700 1 -- start start 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d270c7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20102ac0 0x7f6d201990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d270c7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d20103480 0x7f6d20199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d270c7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d20108a70 0x7f6d2019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d270c7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20112c00 con 0x7f6d20102ac0 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d270c7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20112a80 con 0x7f6d20103480 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d270c7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20112d80 con 0x7f6d20108a70 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d24e63700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20102ac0 0x7f6d201990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d24e63700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20102ac0 0x7f6d201990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35606/0 (socket says 172.21.15.31:35606) 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d24e63700 1 -- 172.21.15.31:0/2150713886 learned_addr learned my addr 172.21.15.31:0/2150713886 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d1ffff700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d20103480 0x7f6d20199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d25664700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d20108a70 0x7f6d2019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d24e63700 1 -- 172.21.15.31:0/2150713886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d20108a70 msgr2=0x7f6d2019d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d24e63700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d20108a70 0x7f6d2019d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d24e63700 1 -- 172.21.15.31:0/2150713886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d20103480 msgr2=0x7f6d20199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d24e63700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6d20103480 0x7f6d20199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.020+0000 7f6d24e63700 1 -- 172.21.15.31:0/2150713886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2019e100 con 0x7f6d20102ac0 2024-04-25T13:18:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.021+0000 7f6d24e63700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20102ac0 0x7f6d201990f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400d950 tx=0x7f6d1400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.021+0000 7f6d1dffb700 1 -- 172.21.15.31:0/2150713886 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14016070 con 0x7f6d20102ac0 2024-04-25T13:18:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.021+0000 7f6d270c7700 1 -- 172.21.15.31:0/2150713886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2019e3f0 con 0x7f6d20102ac0 2024-04-25T13:18:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.021+0000 7f6d1dffb700 1 -- 172.21.15.31:0/2150713886 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d1400fd40 con 0x7f6d20102ac0 2024-04-25T13:18:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.021+0000 7f6d1dffb700 1 -- 172.21.15.31:0/2150713886 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400e3b0 con 0x7f6d20102ac0 2024-04-25T13:18:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.021+0000 7f6d270c7700 1 -- 172.21.15.31:0/2150713886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d201a5c70 con 0x7f6d20102ac0 2024-04-25T13:18:22.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.022+0000 7f6d270c7700 1 -- 172.21.15.31:0/2150713886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d20063190 con 0x7f6d20102ac0 2024-04-25T13:18:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.023+0000 7f6d1dffb700 1 -- 172.21.15.31:0/2150713886 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6d14019070 con 0x7f6d20102ac0 2024-04-25T13:18:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.024+0000 7f6d1dffb700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6d100a3cd0 0x7f6d100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.024+0000 7f6d1ffff700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6d100a3cd0 0x7f6d100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.024+0000 7f6d1dffb700 1 -- 172.21.15.31:0/2150713886 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f6d140d0330 con 0x7f6d20102ac0 2024-04-25T13:18:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.024+0000 7f6d1dffb700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141] conn(0x7f6d100ad4b0 0x7f6d100af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.024+0000 7f6d1dffb700 1 -- 172.21.15.31:0/2150713886 --> [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d100aff80 con 0x7f6d100ad4b0 2024-04-25T13:18:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.024+0000 7f6d1dffb700 1 -- 172.21.15.31:0/2150713886 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f6d140d0810 con 0x7f6d20102ac0 2024-04-25T13:18:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.024+0000 7f6d25664700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141] conn(0x7f6d100ad4b0 0x7f6d100af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.024+0000 7f6d1ffff700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6d100a3cd0 0x7f6d100a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00b5c0 tx=0x7f6d0c005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.025+0000 7f6d25664700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141] conn(0x7f6d100ad4b0 0x7f6d100af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:22.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.025+0000 7f6d1dffb700 1 -- 172.21.15.31:0/2150713886 <== osd.6 v2:172.21.15.31:6818/1320917141 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d100aff80 con 0x7f6d100ad4b0 2024-04-25T13:18:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.042+0000 7f6d270c7700 1 -- 172.21.15.31:0/2150713886 --> [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d20063590 con 0x7f6d100ad4b0 2024-04-25T13:18:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.042+0000 7f6d1dffb700 1 -- 172.21.15.31:0/2150713886 <== osd.6 v2:172.21.15.31:6818/1320917141 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6d20063590 con 0x7f6d100ad4b0 2024-04-25T13:18:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.042+0000 7f6d270c7700 1 -- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141] conn(0x7f6d100ad4b0 msgr2=0x7f6d100af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.042+0000 7f6d270c7700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141] conn(0x7f6d100ad4b0 0x7f6d100af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.042+0000 7f6d270c7700 1 -- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6d100a3cd0 msgr2=0x7f6d100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.042+0000 7f6d270c7700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6d100a3cd0 0x7f6d100a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00b5c0 tx=0x7f6d0c005960 comp rx=0 tx=0).stop 2024-04-25T13:18:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.042+0000 7f6d270c7700 1 -- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20102ac0 msgr2=0x7f6d201990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.042+0000 7f6d270c7700 1 --2- 172.21.15.31:0/2150713886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20102ac0 0x7f6d201990f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400d950 tx=0x7f6d1400dc60 comp rx=0 tx=0).stop 2024-04-25T13:18:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.042+0000 7f6d25664700 1 -- 172.21.15.31:0/2150713886 reap_dead start 2024-04-25T13:18:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.042+0000 7f6d270c7700 1 -- 172.21.15.31:0/2150713886 shutdown_connections 2024-04-25T13:18:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.042+0000 7f6d270c7700 1 -- 172.21.15.31:0/2150713886 >> 172.21.15.31:0/2150713886 conn(0x7f6d20079bb0 msgr2=0x7f6d20101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.043+0000 7f6d270c7700 1 -- 172.21.15.31:0/2150713886 shutdown_connections 2024-04-25T13:18:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.043+0000 7f6d270c7700 1 -- 172.21.15.31:0/2150713886 wait complete. 2024-04-25T13:18:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.234+0000 7f321294a700 1 -- 172.21.15.31:0/3137965825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f320c1019d0 msgr2=0x7f320c10f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.234+0000 7f321294a700 1 --2- 172.21.15.31:0/3137965825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f320c1019d0 0x7f320c10f910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f320400b0a0 tx=0x7f320400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.234+0000 7f321294a700 1 -- 172.21.15.31:0/3137965825 shutdown_connections 2024-04-25T13:18:22.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.234+0000 7f321294a700 1 --2- 172.21.15.31:0/3137965825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f320c1019d0 0x7f320c10f910 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.234+0000 7f321294a700 1 --2- 172.21.15.31:0/3137965825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c108ad0 0x7f320c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.234+0000 7f321294a700 1 --2- 172.21.15.31:0/3137965825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f320c101030 0x7f320c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.234+0000 7f321294a700 1 -- 172.21.15.31:0/3137965825 >> 172.21.15.31:0/3137965825 conn(0x7f320c077fb0 msgr2=0x7f320c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.234+0000 7f321294a700 1 -- 172.21.15.31:0/3137965825 shutdown_connections 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.234+0000 7f321294a700 1 -- 172.21.15.31:0/3137965825 wait complete. 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.235+0000 7f321294a700 1 Processor -- start 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.235+0000 7f321294a700 1 -- start start 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.235+0000 7f321294a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c101030 0x7f320c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.235+0000 7f321294a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f320c1019d0 0x7f320c19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.235+0000 7f321294a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f320c108ad0 0x7f320c19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.235+0000 7f321294a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f320c112c40 con 0x7f320c1019d0 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.235+0000 7f321294a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f320c112ac0 con 0x7f320c101030 2024-04-25T13:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.235+0000 7f321294a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f320c112dc0 con 0x7f320c108ad0 2024-04-25T13:18:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f320b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f320c1019d0 0x7f320c19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f320b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f320c1019d0 0x7f320c19b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35622/0 (socket says 172.21.15.31:35622) 2024-04-25T13:18:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f320b7fe700 1 -- 172.21.15.31:0/2558266506 learned_addr learned my addr 172.21.15.31:0/2558266506 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f320bfff700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c101030 0x7f320c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f3210ee7700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f320c108ad0 0x7f320c19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f320b7fe700 1 -- 172.21.15.31:0/2558266506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f320c108ad0 msgr2=0x7f320c19fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f320b7fe700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f320c108ad0 0x7f320c19fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f320b7fe700 1 -- 172.21.15.31:0/2558266506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c101030 msgr2=0x7f320c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f320b7fe700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f320c101030 0x7f320c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f320b7fe700 1 -- 172.21.15.31:0/2558266506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c1a0280 con 0x7f320c1019d0 2024-04-25T13:18:22.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f320b7fe700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f320c1019d0 0x7f320c19b7b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f320000c9b0 tx=0x7f320000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f32097fa700 1 -- 172.21.15.31:0/2558266506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200007b00 con 0x7f320c1019d0 2024-04-25T13:18:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f32097fa700 1 -- 172.21.15.31:0/2558266506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3200007ca0 con 0x7f320c1019d0 2024-04-25T13:18:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f321294a700 1 -- 172.21.15.31:0/2558266506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320c1a0570 con 0x7f320c1019d0 2024-04-25T13:18:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f32097fa700 1 -- 172.21.15.31:0/2558266506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200018660 con 0x7f320c1019d0 2024-04-25T13:18:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.236+0000 7f321294a700 1 -- 172.21.15.31:0/2558266506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320c1a7df0 con 0x7f320c1019d0 2024-04-25T13:18:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.237+0000 7f321294a700 1 -- 172.21.15.31:0/2558266506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f320c0ffeb0 con 0x7f320c1019d0 2024-04-25T13:18:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.238+0000 7f32097fa700 1 -- 172.21.15.31:0/2558266506 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f320000f450 con 0x7f320c1019d0 2024-04-25T13:18:22.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.239+0000 7f32097fa700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f31f40a3cd0 0x7f31f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.239+0000 7f320bfff700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f31f40a3cd0 0x7f31f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.239+0000 7f32097fa700 1 -- 172.21.15.31:0/2558266506 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f32000d0810 con 0x7f320c1019d0 2024-04-25T13:18:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.239+0000 7f32097fa700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233] conn(0x7f31f40ad4b0 0x7f31f40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.239+0000 7f32097fa700 1 -- 172.21.15.31:0/2558266506 --> [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31f40aff80 con 0x7f31f40ad4b0 2024-04-25T13:18:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.239+0000 7f32097fa700 1 -- 172.21.15.31:0/2558266506 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f32000d0cf0 con 0x7f320c1019d0 2024-04-25T13:18:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.240+0000 7f320bfff700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f31f40a3cd0 0x7f31f40a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009a10 tx=0x7f31fc009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.240+0000 7f3210ee7700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233] conn(0x7f31f40ad4b0 0x7f31f40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.240+0000 7f3210ee7700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233] conn(0x7f31f40ad4b0 0x7f31f40af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.241+0000 7f32097fa700 1 -- 172.21.15.31:0/2558266506 <== osd.4 v2:172.21.15.66:6808/2164191233 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31f40aff80 con 0x7f31f40ad4b0 2024-04-25T13:18:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.259+0000 7f321294a700 1 -- 172.21.15.31:0/2558266506 --> [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f320c104350 con 0x7f31f40ad4b0 2024-04-25T13:18:22.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f32097fa700 1 -- 172.21.15.31:0/2558266506 <== osd.4 v2:172.21.15.66:6808/2164191233 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f320c104350 con 0x7f31f40ad4b0 2024-04-25T13:18:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f321294a700 1 -- 172.21.15.31:0/2558266506 >> [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233] conn(0x7f31f40ad4b0 msgr2=0x7f31f40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f321294a700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233] conn(0x7f31f40ad4b0 0x7f31f40af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f321294a700 1 -- 172.21.15.31:0/2558266506 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f31f40a3cd0 msgr2=0x7f31f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f321294a700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f31f40a3cd0 0x7f31f40a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009a10 tx=0x7f31fc009190 comp rx=0 tx=0).stop 2024-04-25T13:18:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f321294a700 1 -- 172.21.15.31:0/2558266506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f320c1019d0 msgr2=0x7f320c19b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f321294a700 1 --2- 172.21.15.31:0/2558266506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f320c1019d0 0x7f320c19b7b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f320000c9b0 tx=0x7f320000ccc0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f3210ee7700 1 -- 172.21.15.31:0/2558266506 reap_dead start 2024-04-25T13:18:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f321294a700 1 -- 172.21.15.31:0/2558266506 shutdown_connections 2024-04-25T13:18:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f321294a700 1 -- 172.21.15.31:0/2558266506 >> 172.21.15.31:0/2558266506 conn(0x7f320c077fb0 msgr2=0x7f320c0ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f321294a700 1 -- 172.21.15.31:0/2558266506 shutdown_connections 2024-04-25T13:18:22.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.260+0000 7f321294a700 1 -- 172.21.15.31:0/2558266506 wait complete. 2024-04-25T13:18:22.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.568+0000 7f07fccef700 1 -- 172.21.15.31:0/1896036858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07f8102af0 msgr2=0x7f07f8102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.568+0000 7f07fccef700 1 --2- 172.21.15.31:0/1896036858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07f8102af0 0x7f07f8102f70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0099c0 tx=0x7f07ec009cd0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.568+0000 7f07fccef700 1 -- 172.21.15.31:0/1896036858 shutdown_connections 2024-04-25T13:18:22.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.568+0000 7f07fccef700 1 --2- 172.21.15.31:0/1896036858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07f81034b0 0x7f07f810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.568+0000 7f07fccef700 1 --2- 172.21.15.31:0/1896036858 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f8107940 0x7f07f8107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.568+0000 7f07fccef700 1 --2- 172.21.15.31:0/1896036858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07f8102af0 0x7f07f8102f70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.568+0000 7f07fccef700 1 -- 172.21.15.31:0/1896036858 >> 172.21.15.31:0/1896036858 conn(0x7f07f8079ba0 msgr2=0x7f07f8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.568+0000 7f07fccef700 1 -- 172.21.15.31:0/1896036858 shutdown_connections 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.568+0000 7f07fccef700 1 -- 172.21.15.31:0/1896036858 wait complete. 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.569+0000 7f07fccef700 1 Processor -- start 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.569+0000 7f07fccef700 1 -- start start 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.569+0000 7f07fccef700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f8102af0 0x7f07f8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.569+0000 7f07fccef700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07f81034b0 0x7f07f8075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.569+0000 7f07fccef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07f8107940 0x7f07f80757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.569+0000 7f07fccef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f07f8112cb0 con 0x7f07f8107940 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.569+0000 7f07fccef700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f07f8112b30 con 0x7f07f8102af0 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.569+0000 7f07fccef700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07f8112e30 con 0x7f07f81034b0 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.569+0000 7f07f6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07f8107940 0x7f07f80757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.569+0000 7f07f6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07f8107940 0x7f07f80757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35640/0 (socket says 172.21.15.31:35640) 2024-04-25T13:18:22.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.569+0000 7f07f6d9d700 1 -- 172.21.15.31:0/3063804710 learned_addr learned my addr 172.21.15.31:0/3063804710 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:22.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07f5d9b700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07f81034b0 0x7f07f8075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07f659c700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f8102af0 0x7f07f8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07f6d9d700 1 -- 172.21.15.31:0/3063804710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07f81034b0 msgr2=0x7f07f8075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07f6d9d700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07f81034b0 0x7f07f8075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07f6d9d700 1 -- 172.21.15.31:0/3063804710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f8102af0 msgr2=0x7f07f8074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07f6d9d700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07f8102af0 0x7f07f8074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07f6d9d700 1 -- 172.21.15.31:0/3063804710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f81a7670 con 0x7f07f8107940 2024-04-25T13:18:22.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07f6d9d700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07f8107940 0x7f07f80757c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f07f000b9e0 tx=0x7f07f000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:22.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07e77fe700 1 -- 172.21.15.31:0/3063804710 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f000d820 con 0x7f07f8107940 2024-04-25T13:18:22.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07e77fe700 1 -- 172.21.15.31:0/3063804710 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f0015420 con 0x7f07f8107940 2024-04-25T13:18:22.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07fccef700 1 -- 172.21.15.31:0/3063804710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f81a7960 con 0x7f07f8107940 2024-04-25T13:18:22.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07e77fe700 1 -- 172.21.15.31:0/3063804710 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0014610 con 0x7f07f8107940 2024-04-25T13:18:22.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.570+0000 7f07fccef700 1 -- 172.21.15.31:0/3063804710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f81a7e20 con 0x7f07f8107940 2024-04-25T13:18:22.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.571+0000 7f07fccef700 1 -- 172.21.15.31:0/3063804710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07f80630d0 con 0x7f07f8107940 2024-04-25T13:18:22.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.572+0000 7f07e77fe700 1 -- 172.21.15.31:0/3063804710 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f07f00147b0 con 0x7f07f8107940 2024-04-25T13:18:22.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.573+0000 7f07e77fe700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f07e00a3cd0 0x7f07e00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.573+0000 7f07e77fe700 1 -- 172.21.15.31:0/3063804710 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f07f00d0440 con 0x7f07f8107940 2024-04-25T13:18:22.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.573+0000 7f07f659c700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f07e00a3cd0 0x7f07e00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.573+0000 7f07e77fe700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141] conn(0x7f07e00ad4b0 0x7f07e00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.573+0000 7f07e77fe700 1 -- 172.21.15.31:0/3063804710 --> [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07f0012e80 con 0x7f07e00ad4b0 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.573+0000 7f07e77fe700 1 -- 172.21.15.31:0/3063804710 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f07f00d0920 con 0x7f07f8107940 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.573+0000 7f07f659c700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f07e00a3cd0 0x7f07e00a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f07e8005950 tx=0x7f07e800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.573+0000 7f07f5d9b700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141] conn(0x7f07e00ad4b0 0x7f07e00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.574+0000 7f07f5d9b700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141] conn(0x7f07e00ad4b0 0x7f07e00af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.575+0000 7f07e77fe700 1 -- 172.21.15.31:0/3063804710 <== osd.8 v2:172.21.15.136:6816/2823786141 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07f0012e80 con 0x7f07e00ad4b0 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.591+0000 7f07fccef700 1 -- 172.21.15.31:0/3063804710 --> [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f07f80634d0 con 0x7f07e00ad4b0 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.592+0000 7f07e77fe700 1 -- 172.21.15.31:0/3063804710 <== osd.8 v2:172.21.15.136:6816/2823786141 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f07f80634d0 con 0x7f07e00ad4b0 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.592+0000 7f07fccef700 1 -- 172.21.15.31:0/3063804710 >> [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141] conn(0x7f07e00ad4b0 msgr2=0x7f07e00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.592+0000 7f07fccef700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141] conn(0x7f07e00ad4b0 0x7f07e00af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.592+0000 7f07fccef700 1 -- 172.21.15.31:0/3063804710 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f07e00a3cd0 msgr2=0x7f07e00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.592+0000 7f07fccef700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f07e00a3cd0 0x7f07e00a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f07e8005950 tx=0x7f07e800b410 comp rx=0 tx=0).stop 2024-04-25T13:18:22.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.592+0000 7f07fccef700 1 -- 172.21.15.31:0/3063804710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07f8107940 msgr2=0x7f07f80757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:22.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.592+0000 7f07fccef700 1 --2- 172.21.15.31:0/3063804710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07f8107940 0x7f07f80757c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f07f000b9e0 tx=0x7f07f000bcf0 comp rx=0 tx=0).stop 2024-04-25T13:18:22.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.592+0000 7f07f6d9d700 1 -- 172.21.15.31:0/3063804710 reap_dead start 2024-04-25T13:18:22.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.592+0000 7f07fccef700 1 -- 172.21.15.31:0/3063804710 shutdown_connections 2024-04-25T13:18:22.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.592+0000 7f07fccef700 1 -- 172.21.15.31:0/3063804710 >> 172.21.15.31:0/3063804710 conn(0x7f07f8079ba0 msgr2=0x7f07f8100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:22.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.593+0000 7f07fccef700 1 -- 172.21.15.31:0/3063804710 shutdown_connections 2024-04-25T13:18:22.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:22.593+0000 7f07fccef700 1 -- 172.21.15.31:0/3063804710 wait complete. 2024-04-25T13:18:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:22 smithi136 ceph-mon[33038]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:23.230 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:22 smithi031 ceph-mon[29457]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:23.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:22 smithi066 ceph-mon[33137]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:23.630 INFO:teuthology.orchestra.run.smithi031.stdout:150323855418 2024-04-25T13:18:23.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T13:18:24.093 INFO:teuthology.orchestra.run.smithi031.stdout:296352743456 2024-04-25T13:18:24.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T13:18:24.947 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:24 smithi031 ceph-mon[29457]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:18:24.948 INFO:teuthology.orchestra.run.smithi031.stdout:206158430256 2024-04-25T13:18:24.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T13:18:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:24 smithi136 ceph-mon[33038]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:18:25.401 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:25.402 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:25.402 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:24 smithi066 ceph-mon[33137]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:18:25.440 INFO:teuthology.orchestra.run.smithi031.stdout:261993005093 2024-04-25T13:18:25.440 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T13:18:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.724+0000 7fcde7e1f700 1 -- 172.21.15.31:0/2476034218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde0107980 msgr2=0x7fcde0107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.724+0000 7fcde7e1f700 1 --2- 172.21.15.31:0/2476034218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde0107980 0x7fcde0107d60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009990 tx=0x7fcdd0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:25.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.724+0000 7fcde7e1f700 1 -- 172.21.15.31:0/2476034218 shutdown_connections 2024-04-25T13:18:25.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.724+0000 7fcde7e1f700 1 --2- 172.21.15.31:0/2476034218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcde01022f0 0x7fcde010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:25.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.724+0000 7fcde7e1f700 1 --2- 172.21.15.31:0/2476034218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde0107980 0x7fcde0107d60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:25.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.724+0000 7fcde7e1f700 1 --2- 172.21.15.31:0/2476034218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0101930 0x7fcde0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:25.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.724+0000 7fcde7e1f700 1 -- 172.21.15.31:0/2476034218 >> 172.21.15.31:0/2476034218 conn(0x7fcde00788b0 msgr2=0x7fcde0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:25.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.724+0000 7fcde7e1f700 1 -- 172.21.15.31:0/2476034218 shutdown_connections 2024-04-25T13:18:25.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.724+0000 7fcde7e1f700 1 -- 172.21.15.31:0/2476034218 wait complete. 2024-04-25T13:18:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.725+0000 7fcde7e1f700 1 Processor -- start 2024-04-25T13:18:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.725+0000 7fcde7e1f700 1 -- start start 2024-04-25T13:18:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.725+0000 7fcde7e1f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0101930 0x7fcde01034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.725+0000 7fcde7e1f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde01022f0 0x7fcde0103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.725+0000 7fcde7e1f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcde0107980 0x7fcde0103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.725+0000 7fcde7e1f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0111ae0 con 0x7fcde01022f0 2024-04-25T13:18:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.725+0000 7fcde7e1f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0111960 con 0x7fcde0101930 2024-04-25T13:18:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.725+0000 7fcde7e1f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0111c60 con 0x7fcde0107980 2024-04-25T13:18:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.725+0000 7fcde53ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde01022f0 0x7fcde0103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.725+0000 7fcde53ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde01022f0 0x7fcde0103a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35666/0 (socket says 172.21.15.31:35666) 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.725+0000 7fcde53ba700 1 -- 172.21.15.31:0/1554214042 learned_addr learned my addr 172.21.15.31:0/1554214042 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde5bbb700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0101930 0x7fcde01034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde63bc700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcde0107980 0x7fcde0103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde53ba700 1 -- 172.21.15.31:0/1554214042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcde0107980 msgr2=0x7fcde0103f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde53ba700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcde0107980 0x7fcde0103f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde53ba700 1 -- 172.21.15.31:0/1554214042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0101930 msgr2=0x7fcde01034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde53ba700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0101930 0x7fcde01034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde53ba700 1 -- 172.21.15.31:0/1554214042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde01a7710 con 0x7fcde01022f0 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde5bbb700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcde0101930 0x7fcde01034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde63bc700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcde0107980 0x7fcde0103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde53ba700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde01022f0 0x7fcde0103a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00d950 tx=0x7fcddc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcdd6ffd700 1 -- 172.21.15.31:0/1554214042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc016070 con 0x7fcde01022f0 2024-04-25T13:18:25.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde7e1f700 1 -- 172.21.15.31:0/1554214042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde01a7a00 con 0x7fcde01022f0 2024-04-25T13:18:25.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcdd6ffd700 1 -- 172.21.15.31:0/1554214042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcddc00fd40 con 0x7fcde01022f0 2024-04-25T13:18:25.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcde7e1f700 1 -- 172.21.15.31:0/1554214042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde01a7f40 con 0x7fcde01022f0 2024-04-25T13:18:25.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.726+0000 7fcdd6ffd700 1 -- 172.21.15.31:0/1554214042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00e3b0 con 0x7fcde01022f0 2024-04-25T13:18:25.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.727+0000 7fcde7e1f700 1 -- 172.21.15.31:0/1554214042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcde01068c0 con 0x7fcde01022f0 2024-04-25T13:18:25.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.729+0000 7fcdd6ffd700 1 -- 172.21.15.31:0/1554214042 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcddc019070 con 0x7fcde01022f0 2024-04-25T13:18:25.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.729+0000 7fcdd6ffd700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcdcc0a3cd0 0x7fcdcc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:25.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.729+0000 7fcdd6ffd700 1 -- 172.21.15.31:0/1554214042 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fcddc0d0360 con 0x7fcde01022f0 2024-04-25T13:18:25.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.729+0000 7fcde5bbb700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcdcc0a3cd0 0x7fcdcc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:25.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.729+0000 7fcdd6ffd700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911] conn(0x7fcdcc0ad4b0 0x7fcdcc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:25.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.730+0000 7fcdd6ffd700 1 -- 172.21.15.31:0/1554214042 --> [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcdcc0aff80 con 0x7fcdcc0ad4b0 2024-04-25T13:18:25.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.730+0000 7fcdd6ffd700 1 -- 172.21.15.31:0/1554214042 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fcddc0d0840 con 0x7fcde01022f0 2024-04-25T13:18:25.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.730+0000 7fcde5bbb700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcdcc0a3cd0 0x7fcdcc0a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000b5c0 tx=0x7fcdd0005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:25.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.730+0000 7fcde63bc700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911] conn(0x7fcdcc0ad4b0 0x7fcdcc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:25.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.730+0000 7fcde63bc700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911] conn(0x7fcdcc0ad4b0 0x7fcdcc0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:25.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.731+0000 7fcdd6ffd700 1 -- 172.21.15.31:0/1554214042 <== osd.5 v2:172.21.15.136:6808/3732026911 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcdcc0aff80 con 0x7fcdcc0ad4b0 2024-04-25T13:18:25.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.748+0000 7fcde7e1f700 1 -- 172.21.15.31:0/1554214042 --> [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcde0103050 con 0x7fcdcc0ad4b0 2024-04-25T13:18:25.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.748+0000 7fcdd6ffd700 1 -- 172.21.15.31:0/1554214042 <== osd.5 v2:172.21.15.136:6808/3732026911 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcde0103050 con 0x7fcdcc0ad4b0 2024-04-25T13:18:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.749+0000 7fcde7e1f700 1 -- 172.21.15.31:0/1554214042 >> [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911] conn(0x7fcdcc0ad4b0 msgr2=0x7fcdcc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.749+0000 7fcde7e1f700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911] conn(0x7fcdcc0ad4b0 0x7fcdcc0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.749+0000 7fcde7e1f700 1 -- 172.21.15.31:0/1554214042 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcdcc0a3cd0 msgr2=0x7fcdcc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.749+0000 7fcde7e1f700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcdcc0a3cd0 0x7fcdcc0a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000b5c0 tx=0x7fcdd0005960 comp rx=0 tx=0).stop 2024-04-25T13:18:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.749+0000 7fcde7e1f700 1 -- 172.21.15.31:0/1554214042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde01022f0 msgr2=0x7fcde0103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.749+0000 7fcde7e1f700 1 --2- 172.21.15.31:0/1554214042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcde01022f0 0x7fcde0103a20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00d950 tx=0x7fcddc00dc60 comp rx=0 tx=0).stop 2024-04-25T13:18:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.749+0000 7fcde63bc700 1 -- 172.21.15.31:0/1554214042 reap_dead start 2024-04-25T13:18:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.749+0000 7fcde7e1f700 1 -- 172.21.15.31:0/1554214042 shutdown_connections 2024-04-25T13:18:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.749+0000 7fcde7e1f700 1 -- 172.21.15.31:0/1554214042 >> 172.21.15.31:0/1554214042 conn(0x7fcde00788b0 msgr2=0x7fcde006b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.749+0000 7fcde7e1f700 1 -- 172.21.15.31:0/1554214042 shutdown_connections 2024-04-25T13:18:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:25.749+0000 7fcde7e1f700 1 -- 172.21.15.31:0/1554214042 wait complete. 2024-04-25T13:18:26.500 INFO:teuthology.orchestra.run.smithi031.stdout:171798691893 2024-04-25T13:18:26.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T13:18:26.832 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:26.832 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:26.833 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:26.833 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:26.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:26 smithi031 ceph-mon[29457]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:27.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:26 smithi136 ceph-mon[33038]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:26 smithi066 ceph-mon[33137]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:28.938 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:28.939 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:28.939 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:29.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:28 smithi136 ceph-mon[33038]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:29.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:28 smithi031 ceph-mon[29457]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:29.407 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:28 smithi066 ceph-mon[33137]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.903+0000 7f64c5444700 1 -- 172.21.15.31:0/3513415926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0107960 msgr2=0x7f64c0107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.903+0000 7f64c5444700 1 --2- 172.21.15.31:0/3513415926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0107960 0x7f64c0107d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009990 tx=0x7f64b0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:30.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.903+0000 7f64c5444700 1 -- 172.21.15.31:0/3513415926 shutdown_connections 2024-04-25T13:18:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.903+0000 7f64c5444700 1 --2- 172.21.15.31:0/3513415926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64c0102320 0x7f64c010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.903+0000 7f64c5444700 1 --2- 172.21.15.31:0/3513415926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0107960 0x7f64c0107d40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.903+0000 7f64c5444700 1 --2- 172.21.15.31:0/3513415926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c0101960 0x7f64c0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.903+0000 7f64c5444700 1 -- 172.21.15.31:0/3513415926 >> 172.21.15.31:0/3513415926 conn(0x7f64c00789e0 msgr2=0x7f64c0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.903+0000 7f64c5444700 1 -- 172.21.15.31:0/3513415926 shutdown_connections 2024-04-25T13:18:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.903+0000 7f64c5444700 1 -- 172.21.15.31:0/3513415926 wait complete. 2024-04-25T13:18:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.903+0000 7f64c5444700 1 Processor -- start 2024-04-25T13:18:30.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64c5444700 1 -- start start 2024-04-25T13:18:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64c5444700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c0101960 0x7f64c0110d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64c5444700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0102320 0x7f64c01112b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64c5444700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64c0107960 0x7f64c01117f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64c5444700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f64c006c070 con 0x7f64c0102320 2024-04-25T13:18:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64c5444700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64c006bef0 con 0x7f64c0101960 2024-04-25T13:18:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64c5444700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f64c0112eb0 con 0x7f64c0107960 2024-04-25T13:18:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64be7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0102320 0x7f64c01112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:30.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64be7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0102320 0x7f64c01112b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35690/0 (socket says 172.21.15.31:35690) 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64be7fc700 1 -- 172.21.15.31:0/1081419191 learned_addr learned my addr 172.21.15.31:0/1081419191 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64bf7fe700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64c0107960 0x7f64c01117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64beffd700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c0101960 0x7f64c0110d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64be7fc700 1 -- 172.21.15.31:0/1081419191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64c0107960 msgr2=0x7f64c01117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64be7fc700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64c0107960 0x7f64c01117f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64be7fc700 1 -- 172.21.15.31:0/1081419191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c0101960 msgr2=0x7f64c0110d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64be7fc700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c0101960 0x7f64c0110d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64be7fc700 1 -- 172.21.15.31:0/1081419191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c01a7640 con 0x7f64c0102320 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64beffd700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c0101960 0x7f64c0110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.904+0000 7f64be7fc700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0102320 0x7f64c01112b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f64b400b530 tx=0x7f64b400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.905+0000 7f64a7fff700 1 -- 172.21.15.31:0/1081419191 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4015020 con 0x7f64c0102320 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.905+0000 7f64c5444700 1 -- 172.21.15.31:0/1081419191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c01a7930 con 0x7f64c0102320 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.905+0000 7f64a7fff700 1 -- 172.21.15.31:0/1081419191 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b4011ce0 con 0x7f64c0102320 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.905+0000 7f64a7fff700 1 -- 172.21.15.31:0/1081419191 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4010e40 con 0x7f64c0102320 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.905+0000 7f64c5444700 1 -- 172.21.15.31:0/1081419191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c01a7e70 con 0x7f64c0102320 2024-04-25T13:18:30.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.906+0000 7f64c5444700 1 -- 172.21.15.31:0/1081419191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c0069730 con 0x7f64c0102320 2024-04-25T13:18:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.907+0000 7f64a7fff700 1 -- 172.21.15.31:0/1081419191 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f64b4018020 con 0x7f64c0102320 2024-04-25T13:18:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.908+0000 7f64a7fff700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f64a80a3cd0 0x7f64a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.908+0000 7f64beffd700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f64a80a3cd0 0x7f64a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:30.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.908+0000 7f64a7fff700 1 -- 172.21.15.31:0/1081419191 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f64b40d0080 con 0x7f64c0102320 2024-04-25T13:18:30.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.908+0000 7f64beffd700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f64a80a3cd0 0x7f64a80a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f64b0000c00 tx=0x7f64b0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:30.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:30.912+0000 7f64a7fff700 1 -- 172.21.15.31:0/1081419191 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f64b401f020 con 0x7f64c0102320 2024-04-25T13:18:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.122+0000 7f64c5444700 1 -- 172.21.15.31:0/1081419191 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f64c01a81a0 con 0x7f64c0102320 2024-04-25T13:18:31.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:30 smithi031 ceph-mon[29457]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.122+0000 7f64a7fff700 1 -- 172.21.15.31:0/1081419191 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f64b409cbc0 con 0x7f64c0102320 2024-04-25T13:18:31.123 INFO:teuthology.orchestra.run.smithi031.stdout:34359738447 2024-04-25T13:18:31.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 -- 172.21.15.31:0/1081419191 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f64a80a3cd0 msgr2=0x7f64a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:31.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f64a80a3cd0 0x7f64a80a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f64b0000c00 tx=0x7f64b0005fb0 comp rx=0 tx=0).stop 2024-04-25T13:18:31.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 -- 172.21.15.31:0/1081419191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0102320 msgr2=0x7f64c01112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:31.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0102320 0x7f64c01112b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f64b400b530 tx=0x7f64b400b840 comp rx=0 tx=0).stop 2024-04-25T13:18:31.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 -- 172.21.15.31:0/1081419191 shutdown_connections 2024-04-25T13:18:31.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f64a80a3cd0 0x7f64a80a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:31.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64c0107960 0x7f64c01117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:31.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64c0101960 0x7f64c0110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:31.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 --2- 172.21.15.31:0/1081419191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64c0102320 0x7f64c01112b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:31.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 -- 172.21.15.31:0/1081419191 >> 172.21.15.31:0/1081419191 conn(0x7f64c00789e0 msgr2=0x7f64c01043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:31.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 -- 172.21.15.31:0/1081419191 shutdown_connections 2024-04-25T13:18:31.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:31.125+0000 7f64c5444700 1 -- 172.21.15.31:0/1081419191 wait complete. 2024-04-25T13:18:31.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:30 smithi136 ceph-mon[33038]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:30 smithi066 ceph-mon[33137]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:32.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:31 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1081419191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:18:32.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T13:18:32.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T13:18:32.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-04-25T13:18:32.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-25T13:18:32.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T13:18:32.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T13:18:32.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T13:18:32.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T13:18:32.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]: dispatch 2024-04-25T13:18:32.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:31 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 8]}]: dispatch 2024-04-25T13:18:32.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:31 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1081419191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:18:32.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T13:18:32.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T13:18:32.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-04-25T13:18:32.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-25T13:18:32.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T13:18:32.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T13:18:32.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T13:18:32.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T13:18:32.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]: dispatch 2024-04-25T13:18:32.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:31 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 8]}]: dispatch 2024-04-25T13:18:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:31 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1081419191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:18:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-04-25T13:18:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T13:18:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-04-25T13:18:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-25T13:18:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T13:18:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T13:18:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T13:18:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T13:18:32.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]: dispatch 2024-04-25T13:18:32.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:31 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 8]}]: dispatch 2024-04-25T13:18:32.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.611+0000 7f23313f5700 1 -- 172.21.15.31:0/848533069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f232c102ac0 msgr2=0x7f232c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:32.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.611+0000 7f23313f5700 1 --2- 172.21.15.31:0/848533069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f232c102ac0 0x7f232c102f40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2320009990 tx=0x7f2320009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:32.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.611+0000 7f23313f5700 1 -- 172.21.15.31:0/848533069 shutdown_connections 2024-04-25T13:18:32.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.611+0000 7f23313f5700 1 --2- 172.21.15.31:0/848533069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f232c103480 0x7f232c10f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:32.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.611+0000 7f23313f5700 1 --2- 172.21.15.31:0/848533069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c108ab0 0x7f232c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:32.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.611+0000 7f23313f5700 1 --2- 172.21.15.31:0/848533069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f232c102ac0 0x7f232c102f40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:32.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.611+0000 7f23313f5700 1 -- 172.21.15.31:0/848533069 >> 172.21.15.31:0/848533069 conn(0x7f232c079bf0 msgr2=0x7f232c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:32.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.611+0000 7f23313f5700 1 -- 172.21.15.31:0/848533069 shutdown_connections 2024-04-25T13:18:32.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.611+0000 7f23313f5700 1 -- 172.21.15.31:0/848533069 wait complete. 2024-04-25T13:18:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f23313f5700 1 Processor -- start 2024-04-25T13:18:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f23313f5700 1 -- start start 2024-04-25T13:18:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f23313f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f232c102ac0 0x7f232c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f23313f5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f232c103480 0x7f232c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f23313f5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c108ab0 0x7f232c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f23313f5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f232c112d50 con 0x7f232c102ac0 2024-04-25T13:18:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f23313f5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f232c112bd0 con 0x7f232c108ab0 2024-04-25T13:18:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f23313f5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f232c112ed0 con 0x7f232c103480 2024-04-25T13:18:32.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f232affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f232c102ac0 0x7f232c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f232affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f232c102ac0 0x7f232c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39604/0 (socket says 172.21.15.31:39604) 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f232affd700 1 -- 172.21.15.31:0/1038683550 learned_addr learned my addr 172.21.15.31:0/1038683550 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f232a7fc700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f232c103480 0x7f232c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f232b7fe700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c108ab0 0x7f232c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f232affd700 1 -- 172.21.15.31:0/1038683550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f232c103480 msgr2=0x7f232c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f232affd700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f232c103480 0x7f232c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f232affd700 1 -- 172.21.15.31:0/1038683550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c108ab0 msgr2=0x7f232c19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.612+0000 7f232affd700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c108ab0 0x7f232c19dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.613+0000 7f232affd700 1 -- 172.21.15.31:0/1038683550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c19e220 con 0x7f232c102ac0 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.613+0000 7f232affd700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f232c102ac0 0x7f232c199260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f231c00ba50 tx=0x7f231c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.613+0000 7f2313fff700 1 -- 172.21.15.31:0/1038683550 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c019070 con 0x7f232c102ac0 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.613+0000 7f2313fff700 1 -- 172.21.15.31:0/1038683550 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f231c014440 con 0x7f232c102ac0 2024-04-25T13:18:32.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.613+0000 7f23313f5700 1 -- 172.21.15.31:0/1038683550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c19e510 con 0x7f232c102ac0 2024-04-25T13:18:32.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.613+0000 7f2313fff700 1 -- 172.21.15.31:0/1038683550 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c00ce60 con 0x7f232c102ac0 2024-04-25T13:18:32.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.613+0000 7f23313f5700 1 -- 172.21.15.31:0/1038683550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c1a5d90 con 0x7f232c102ac0 2024-04-25T13:18:32.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.614+0000 7f23313f5700 1 -- 172.21.15.31:0/1038683550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c1041f0 con 0x7f232c102ac0 2024-04-25T13:18:32.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.615+0000 7f2313fff700 1 -- 172.21.15.31:0/1038683550 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f231c01c070 con 0x7f232c102ac0 2024-04-25T13:18:32.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.616+0000 7f2313fff700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23140a3cd0 0x7f23140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:32.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.616+0000 7f232a7fc700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23140a3cd0 0x7f23140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:32.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.616+0000 7f2313fff700 1 -- 172.21.15.31:0/1038683550 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8269+0+0 (secure 0 0 0) 0x7f231c0cfdb0 con 0x7f232c102ac0 2024-04-25T13:18:32.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.616+0000 7f232a7fc700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23140a3cd0 0x7f23140a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f232000b5c0 tx=0x7f2320005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:32.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.619+0000 7f2313fff700 1 -- 172.21.15.31:0/1038683550 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f231c09c6c0 con 0x7f232c102ac0 2024-04-25T13:18:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.833+0000 7f23313f5700 1 -- 172.21.15.31:0/1038683550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f232c19a570 con 0x7f232c102ac0 2024-04-25T13:18:32.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.833+0000 7f2313fff700 1 -- 172.21.15.31:0/1038683550 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f231c09bfc0 con 0x7f232c102ac0 2024-04-25T13:18:32.834 INFO:teuthology.orchestra.run.smithi031.stdout:171798691894 2024-04-25T13:18:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.835+0000 7f23313f5700 1 -- 172.21.15.31:0/1038683550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23140a3cd0 msgr2=0x7f23140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.835+0000 7f23313f5700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23140a3cd0 0x7f23140a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f232000b5c0 tx=0x7f2320005960 comp rx=0 tx=0).stop 2024-04-25T13:18:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.835+0000 7f23313f5700 1 -- 172.21.15.31:0/1038683550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f232c102ac0 msgr2=0x7f232c199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.835+0000 7f23313f5700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f232c102ac0 0x7f232c199260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f231c00ba50 tx=0x7f231c00bd60 comp rx=0 tx=0).stop 2024-04-25T13:18:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.835+0000 7f23313f5700 1 -- 172.21.15.31:0/1038683550 shutdown_connections 2024-04-25T13:18:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.835+0000 7f23313f5700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f232c108ab0 0x7f232c19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.836+0000 7f23313f5700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f232c102ac0 0x7f232c199260 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.836+0000 7f23313f5700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23140a3cd0 0x7f23140a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.836+0000 7f23313f5700 1 --2- 172.21.15.31:0/1038683550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f232c103480 0x7f232c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.836+0000 7f23313f5700 1 -- 172.21.15.31:0/1038683550 >> 172.21.15.31:0/1038683550 conn(0x7f232c079bf0 msgr2=0x7f232c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:32.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.836+0000 7f23313f5700 1 -- 172.21.15.31:0/1038683550 shutdown_connections 2024-04-25T13:18:32.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:32.836+0000 7f23313f5700 1 -- 172.21.15.31:0/1038683550 wait complete. 2024-04-25T13:18:33.159 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:33.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T13:18:33.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T13:18:33.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-04-25T13:18:33.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-25T13:18:33.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T13:18:33.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T13:18:33.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T13:18:33.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T13:18:33.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]': finished 2024-04-25T13:18:33.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 8]}]': finished 2024-04-25T13:18:33.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:18:33.161 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:32 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1038683550' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:18:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T13:18:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T13:18:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-04-25T13:18:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-25T13:18:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T13:18:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T13:18:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T13:18:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T13:18:33.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]': finished 2024-04-25T13:18:33.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 8]}]': finished 2024-04-25T13:18:33.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:18:33.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:32 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1038683550' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:18:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-04-25T13:18:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T13:18:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-04-25T13:18:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-25T13:18:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T13:18:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T13:18:33.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T13:18:33.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T13:18:33.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]': finished 2024-04-25T13:18:33.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2e", "id": [1, 8]}]': finished 2024-04-25T13:18:33.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:18:33.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:32 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1038683550' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:18:34.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:33 smithi031 ceph-mon[29457]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:18:34.103 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691893 got 171798691894 for osd.5 2024-04-25T13:18:34.103 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:34.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:33 smithi136 ceph-mon[33038]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:18:34.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:33 smithi066 ceph-mon[33137]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:18:35.206 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:34 smithi031 ceph-mon[29457]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:35.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:34 smithi136 ceph-mon[33038]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:35.240 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738447 for osd.0 2024-04-25T13:18:35.240 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:34 smithi066 ceph-mon[33137]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:36.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.021+0000 7f5edad75700 1 -- 172.21.15.31:0/4195310509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed406b1f0 msgr2=0x7f5ed406b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.021+0000 7f5edad75700 1 --2- 172.21.15.31:0/4195310509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed406b1f0 0x7f5ed406b670 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009990 tx=0x7f5ec0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.021+0000 7f5edad75700 1 -- 172.21.15.31:0/4195310509 shutdown_connections 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.021+0000 7f5edad75700 1 --2- 172.21.15.31:0/4195310509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed4108a60 0x7f5ed4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.021+0000 7f5edad75700 1 --2- 172.21.15.31:0/4195310509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed406bbb0 0x7f5ed410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.021+0000 7f5edad75700 1 --2- 172.21.15.31:0/4195310509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed406b1f0 0x7f5ed406b670 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.021+0000 7f5edad75700 1 -- 172.21.15.31:0/4195310509 >> 172.21.15.31:0/4195310509 conn(0x7f5ed40789e0 msgr2=0x7f5ed4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.021+0000 7f5edad75700 1 -- 172.21.15.31:0/4195310509 shutdown_connections 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.021+0000 7f5edad75700 1 -- 172.21.15.31:0/4195310509 wait complete. 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.021+0000 7f5edad75700 1 Processor -- start 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.021+0000 7f5edad75700 1 -- start start 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5edad75700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed406b1f0 0x7f5ed41990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5edad75700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed406bbb0 0x7f5ed4199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5edad75700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed4108a60 0x7f5ed419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:36.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5edad75700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4112cf0 con 0x7f5ed4108a60 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5edad75700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4112b70 con 0x7f5ed406b1f0 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5edad75700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4112e70 con 0x7f5ed406bbb0 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5ed8b11700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed406b1f0 0x7f5ed41990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5ed8b11700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed406b1f0 0x7f5ed41990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.31:34732/0 (socket says 172.21.15.31:34732) 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5ed8b11700 1 -- 172.21.15.31:0/1180032187 learned_addr learned my addr 172.21.15.31:0/1180032187 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5ed3fff700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed406bbb0 0x7f5ed4199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5ed9312700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed4108a60 0x7f5ed419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5ed8b11700 1 -- 172.21.15.31:0/1180032187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed406bbb0 msgr2=0x7f5ed4199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5ed8b11700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed406bbb0 0x7f5ed4199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5ed8b11700 1 -- 172.21.15.31:0/1180032187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed4108a60 msgr2=0x7f5ed419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5ed8b11700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed4108a60 0x7f5ed419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.022+0000 7f5ed8b11700 1 -- 172.21.15.31:0/1180032187 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed419e080 con 0x7f5ed406b1f0 2024-04-25T13:18:36.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.023+0000 7f5ed8b11700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed406b1f0 0x7f5ed41990c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800adc0 tx=0x7f5ec80074a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.023+0000 7f5ed1ffb700 1 -- 172.21.15.31:0/1180032187 <== mon.2 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec801d070 con 0x7f5ed406b1f0 2024-04-25T13:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.023+0000 7f5edad75700 1 -- 172.21.15.31:0/1180032187 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed419e370 con 0x7f5ed406b1f0 2024-04-25T13:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.023+0000 7f5ed1ffb700 1 -- 172.21.15.31:0/1180032187 <== mon.2 v2:172.21.15.66:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec8007c50 con 0x7f5ed406b1f0 2024-04-25T13:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.024+0000 7f5ed1ffb700 1 -- 172.21.15.31:0/1180032187 <== mon.2 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8017610 con 0x7f5ed406b1f0 2024-04-25T13:18:36.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.024+0000 7f5edad75700 1 -- 172.21.15.31:0/1180032187 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41018e0 con 0x7f5ed406b1f0 2024-04-25T13:18:36.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.025+0000 7f5edad75700 1 -- 172.21.15.31:0/1180032187 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed4069730 con 0x7f5ed406b1f0 2024-04-25T13:18:36.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.026+0000 7f5ed1ffb700 1 -- 172.21.15.31:0/1180032187 <== mon.2 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ec8018410 con 0x7f5ed406b1f0 2024-04-25T13:18:36.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.026+0000 7f5ed1ffb700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5ec40a3cd0 0x7f5ec40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:36.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.026+0000 7f5ed3fff700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5ec40a3cd0 0x7f5ec40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:36.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.026+0000 7f5ed1ffb700 1 -- 172.21.15.31:0/1180032187 <== mon.2 v2:172.21.15.66:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f5ec80d3a70 con 0x7f5ed406b1f0 2024-04-25T13:18:36.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.027+0000 7f5ed3fff700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5ec40a3cd0 0x7f5ec40a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4079050 tx=0x7f5ec000b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:36.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.031+0000 7f5ed1ffb700 1 -- 172.21.15.31:0/1180032187 <== mon.2 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ec8053ff0 con 0x7f5ed406b1f0 2024-04-25T13:18:36.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.246+0000 7f5edad75700 1 -- 172.21.15.31:0/1180032187 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5ed402df80 con 0x7f5ed406b1f0 2024-04-25T13:18:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.247+0000 7f5ed1ffb700 1 -- 172.21.15.31:0/1180032187 <== mon.2 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5ec80a02d0 con 0x7f5ed406b1f0 2024-04-25T13:18:36.248 INFO:teuthology.orchestra.run.smithi031.stdout:55834574922 2024-04-25T13:18:36.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 -- 172.21.15.31:0/1180032187 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5ec40a3cd0 msgr2=0x7f5ec40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:36.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5ec40a3cd0 0x7f5ec40a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4079050 tx=0x7f5ec000b540 comp rx=0 tx=0).stop 2024-04-25T13:18:36.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 -- 172.21.15.31:0/1180032187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed406b1f0 msgr2=0x7f5ed41990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:36.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed406b1f0 0x7f5ed41990c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800adc0 tx=0x7f5ec80074a0 comp rx=0 tx=0).stop 2024-04-25T13:18:36.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 -- 172.21.15.31:0/1180032187 shutdown_connections 2024-04-25T13:18:36.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ed4108a60 0x7f5ed419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:36.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ed406b1f0 0x7f5ed41990c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:36.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f5ec40a3cd0 0x7f5ec40a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:36.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 --2- 172.21.15.31:0/1180032187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed406bbb0 0x7f5ed4199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:36.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 -- 172.21.15.31:0/1180032187 >> 172.21.15.31:0/1180032187 conn(0x7f5ed40789e0 msgr2=0x7f5ed40ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:36.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 -- 172.21.15.31:0/1180032187 shutdown_connections 2024-04-25T13:18:36.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:36.249+0000 7f5edad75700 1 -- 172.21.15.31:0/1180032187 wait complete. 2024-04-25T13:18:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:36 smithi031 ceph-mon[29457]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:36 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1180032187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:18:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:36 smithi136 ceph-mon[33038]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:36 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1180032187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:18:37.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:36 smithi066 ceph-mon[33137]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:37.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:36 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1180032187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:18:37.980 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:37.981 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:38.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.852+0000 7f9cb0f14700 1 -- 172.21.15.31:0/1530402289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac102ac0 msgr2=0x7f9cac102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.852+0000 7f9cb0f14700 1 --2- 172.21.15.31:0/1530402289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac102ac0 0x7f9cac102f40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009990 tx=0x7f9ca0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.852+0000 7f9cb0f14700 1 -- 172.21.15.31:0/1530402289 shutdown_connections 2024-04-25T13:18:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.852+0000 7f9cb0f14700 1 --2- 172.21.15.31:0/1530402289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cac108a70 0x7f9cac108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.852+0000 7f9cb0f14700 1 --2- 172.21.15.31:0/1530402289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cac103480 0x7f9cac10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.852+0000 7f9cb0f14700 1 --2- 172.21.15.31:0/1530402289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac102ac0 0x7f9cac102f40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.852+0000 7f9cb0f14700 1 -- 172.21.15.31:0/1530402289 >> 172.21.15.31:0/1530402289 conn(0x7f9cac079bb0 msgr2=0x7f9cac079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.852+0000 7f9cb0f14700 1 -- 172.21.15.31:0/1530402289 shutdown_connections 2024-04-25T13:18:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.852+0000 7f9cb0f14700 1 -- 172.21.15.31:0/1530402289 wait complete. 2024-04-25T13:18:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9cb0f14700 1 Processor -- start 2024-04-25T13:18:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9cb0f14700 1 -- start start 2024-04-25T13:18:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9cb0f14700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac102ac0 0x7f9cac199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9cb0f14700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cac103480 0x7f9cac1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9cb0f14700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cac108a70 0x7f9cac19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9cb0f14700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac112d00 con 0x7f9cac102ac0 2024-04-25T13:18:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9cb0f14700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac112b80 con 0x7f9cac103480 2024-04-25T13:18:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9cb0f14700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac112e80 con 0x7f9cac108a70 2024-04-25T13:18:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9caa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac102ac0 0x7f9cac199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9caa59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac102ac0 0x7f9cac199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39650/0 (socket says 172.21.15.31:39650) 2024-04-25T13:18:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9caa59c700 1 -- 172.21.15.31:0/240106151 learned_addr learned my addr 172.21.15.31:0/240106151 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9ca9d9b700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cac103480 0x7f9cac1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9caad9d700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cac108a70 0x7f9cac19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9caa59c700 1 -- 172.21.15.31:0/240106151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cac108a70 msgr2=0x7f9cac19db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9caa59c700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cac108a70 0x7f9cac19db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9caa59c700 1 -- 172.21.15.31:0/240106151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cac103480 msgr2=0x7f9cac1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9caa59c700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cac103480 0x7f9cac1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.853+0000 7f9caa59c700 1 -- 172.21.15.31:0/240106151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac19e280 con 0x7f9cac102ac0 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.854+0000 7f9caa59c700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac102ac0 0x7f9cac199270 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00ba50 tx=0x7f9c9c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.854+0000 7f9c9b7fe700 1 -- 172.21.15.31:0/240106151 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c019070 con 0x7f9cac102ac0 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.854+0000 7f9cb0f14700 1 -- 172.21.15.31:0/240106151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac19e570 con 0x7f9cac102ac0 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.854+0000 7f9c9b7fe700 1 -- 172.21.15.31:0/240106151 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c9c00cde0 con 0x7f9cac102ac0 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.854+0000 7f9c9b7fe700 1 -- 172.21.15.31:0/240106151 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c007370 con 0x7f9cac102ac0 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.854+0000 7f9cb0f14700 1 -- 172.21.15.31:0/240106151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac1a5df0 con 0x7f9cac102ac0 2024-04-25T13:18:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.855+0000 7f9cb0f14700 1 -- 172.21.15.31:0/240106151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cac069730 con 0x7f9cac102ac0 2024-04-25T13:18:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.856+0000 7f9c9b7fe700 1 -- 172.21.15.31:0/240106151 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9c9c01c070 con 0x7f9cac102ac0 2024-04-25T13:18:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.857+0000 7f9c9b7fe700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9c940a3cd0 0x7f9c940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.857+0000 7f9ca9d9b700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9c940a3cd0 0x7f9c940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.857+0000 7f9c9b7fe700 1 -- 172.21.15.31:0/240106151 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f9c9c0d0100 con 0x7f9cac102ac0 2024-04-25T13:18:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.857+0000 7f9ca9d9b700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9c940a3cd0 0x7f9c940a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000b5c0 tx=0x7f9ca0005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:38.860+0000 7f9c9b7fe700 1 -- 172.21.15.31:0/240106151 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c9c023070 con 0x7f9cac102ac0 2024-04-25T13:18:39.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.076+0000 7f9cb0f14700 1 -- 172.21.15.31:0/240106151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9cac02df80 con 0x7f9cac102ac0 2024-04-25T13:18:39.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.077+0000 7f9c9b7fe700 1 -- 172.21.15.31:0/240106151 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9c9c09ca10 con 0x7f9cac102ac0 2024-04-25T13:18:39.078 INFO:teuthology.orchestra.run.smithi031.stdout:115964117058 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 -- 172.21.15.31:0/240106151 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9c940a3cd0 msgr2=0x7f9c940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9c940a3cd0 0x7f9c940a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000b5c0 tx=0x7f9ca0005960 comp rx=0 tx=0).stop 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 -- 172.21.15.31:0/240106151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac102ac0 msgr2=0x7f9cac199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac102ac0 0x7f9cac199270 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00ba50 tx=0x7f9c9c00bd60 comp rx=0 tx=0).stop 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 -- 172.21.15.31:0/240106151 shutdown_connections 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f9c940a3cd0 0x7f9c940a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cac108a70 0x7f9cac19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cac102ac0 0x7f9cac199270 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 --2- 172.21.15.31:0/240106151 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9cac103480 0x7f9cac1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 -- 172.21.15.31:0/240106151 >> 172.21.15.31:0/240106151 conn(0x7f9cac079bb0 msgr2=0x7f9cac1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 -- 172.21.15.31:0/240106151 shutdown_connections 2024-04-25T13:18:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.079+0000 7f9cb0f14700 1 -- 172.21.15.31:0/240106151 wait complete. 2024-04-25T13:18:39.218 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:38 smithi031 ceph-mon[29457]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-25T13:18:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:38 smithi136 ceph-mon[33038]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-25T13:18:39.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:38 smithi066 ceph-mon[33137]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-25T13:18:39.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.635+0000 7ff1a2782700 1 -- 172.21.15.31:0/778928347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff19c075590 msgr2=0x7ff19c10ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:39.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.635+0000 7ff1a2782700 1 --2- 172.21.15.31:0/778928347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff19c075590 0x7ff19c10ef00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff184009990 tx=0x7ff184009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.635+0000 7ff1a2782700 1 -- 172.21.15.31:0/778928347 shutdown_connections 2024-04-25T13:18:39.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.635+0000 7ff1a2782700 1 --2- 172.21.15.31:0/778928347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff19c074c70 0x7ff19c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.635+0000 7ff1a2782700 1 --2- 172.21.15.31:0/778928347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff19c10f440 0x7ff19c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.635+0000 7ff1a2782700 1 --2- 172.21.15.31:0/778928347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff19c075590 0x7ff19c10ef00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.635+0000 7ff1a2782700 1 -- 172.21.15.31:0/778928347 >> 172.21.15.31:0/778928347 conn(0x7ff19c079ba0 msgr2=0x7ff19c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.635+0000 7ff1a2782700 1 -- 172.21.15.31:0/778928347 shutdown_connections 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.635+0000 7ff1a2782700 1 -- 172.21.15.31:0/778928347 wait complete. 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.635+0000 7ff1a2782700 1 Processor -- start 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.635+0000 7ff1a2782700 1 -- start start 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff1a2782700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff19c074c70 0x7ff19c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff1a2782700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff19c075590 0x7ff19c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff1a2782700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff19c10f440 0x7ff19c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff1a2782700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c114ec0 con 0x7ff19c075590 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff1a2782700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c114d40 con 0x7ff19c074c70 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff1a2782700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c115040 con 0x7ff19c10f440 2024-04-25T13:18:39.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff19b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff19c075590 0x7ff19c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff19b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff19c075590 0x7ff19c19b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39670/0 (socket says 172.21.15.31:39670) 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff19b7fe700 1 -- 172.21.15.31:0/50047405 learned_addr learned my addr 172.21.15.31:0/50047405 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff19bfff700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff19c074c70 0x7ff19c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff1a0d1f700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff19c10f440 0x7ff19c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff19b7fe700 1 -- 172.21.15.31:0/50047405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff19c10f440 msgr2=0x7ff19c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff19b7fe700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff19c10f440 0x7ff19c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff19b7fe700 1 -- 172.21.15.31:0/50047405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff19c074c70 msgr2=0x7ff19c19b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff19b7fe700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff19c074c70 0x7ff19c19b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff19b7fe700 1 -- 172.21.15.31:0/50047405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c1a0410 con 0x7ff19c075590 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff19bfff700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff19c074c70 0x7ff19c19b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff19b7fe700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff19c075590 0x7ff19c19b990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff18400b5c0 tx=0x7ff1840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.636+0000 7ff1997fa700 1 -- 172.21.15.31:0/50047405 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184027070 con 0x7ff19c075590 2024-04-25T13:18:39.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.637+0000 7ff1a2782700 1 -- 172.21.15.31:0/50047405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c1a06a0 con 0x7ff19c075590 2024-04-25T13:18:39.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.637+0000 7ff1997fa700 1 -- 172.21.15.31:0/50047405 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff184003f20 con 0x7ff19c075590 2024-04-25T13:18:39.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.637+0000 7ff1997fa700 1 -- 172.21.15.31:0/50047405 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184022c80 con 0x7ff19c075590 2024-04-25T13:18:39.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.637+0000 7ff1a2782700 1 -- 172.21.15.31:0/50047405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c1a7f80 con 0x7ff19c075590 2024-04-25T13:18:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.638+0000 7ff1a2782700 1 -- 172.21.15.31:0/50047405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19c069730 con 0x7ff19c075590 2024-04-25T13:18:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.639+0000 7ff1997fa700 1 -- 172.21.15.31:0/50047405 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff18402a070 con 0x7ff19c075590 2024-04-25T13:18:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.640+0000 7ff1997fa700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1880a3cd0 0x7ff1880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.640+0000 7ff19bfff700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1880a3cd0 0x7ff1880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.640+0000 7ff1997fa700 1 -- 172.21.15.31:0/50047405 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7ff1840d17a0 con 0x7ff19c075590 2024-04-25T13:18:39.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.640+0000 7ff19bfff700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1880a3cd0 0x7ff1880a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff18c0099d0 tx=0x7ff18c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:39.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.643+0000 7ff1997fa700 1 -- 172.21.15.31:0/50047405 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff184013070 con 0x7ff19c075590 2024-04-25T13:18:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.871+0000 7ff1a2782700 1 -- 172.21.15.31:0/50047405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff19c02df80 con 0x7ff19c075590 2024-04-25T13:18:39.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.871+0000 7ff1997fa700 1 -- 172.21.15.31:0/50047405 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff18409e030 con 0x7ff19c075590 2024-04-25T13:18:39.873 INFO:teuthology.orchestra.run.smithi031.stdout:77309411399 2024-04-25T13:18:39.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 -- 172.21.15.31:0/50047405 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1880a3cd0 msgr2=0x7ff1880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:39.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1880a3cd0 0x7ff1880a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff18c0099d0 tx=0x7ff18c008040 comp rx=0 tx=0).stop 2024-04-25T13:18:39.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 -- 172.21.15.31:0/50047405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff19c075590 msgr2=0x7ff19c19b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:39.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff19c075590 0x7ff19c19b990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff18400b5c0 tx=0x7ff1840047b0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 -- 172.21.15.31:0/50047405 shutdown_connections 2024-04-25T13:18:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1880a3cd0 0x7ff1880a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff19c10f440 0x7ff19c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff19c074c70 0x7ff19c19b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 --2- 172.21.15.31:0/50047405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff19c075590 0x7ff19c19b990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 -- 172.21.15.31:0/50047405 >> 172.21.15.31:0/50047405 conn(0x7ff19c079ba0 msgr2=0x7ff19c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 -- 172.21.15.31:0/50047405 shutdown_connections 2024-04-25T13:18:39.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:39.874+0000 7ff1a2782700 1 -- 172.21.15.31:0/50047405 wait complete. 2024-04-25T13:18:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:39 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/240106151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:18:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:39 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:39 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/50047405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:18:40.219 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:39 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/240106151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:18:40.219 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:39 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:40.219 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:39 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/50047405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:18:40.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:39 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/240106151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:18:40.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:39 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:40.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:39 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/50047405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:18:40.844 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411399 for osd.2 2024-04-25T13:18:40.844 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:41.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.057+0000 7f183be70700 1 -- 172.21.15.31:0/2110071823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1834075590 msgr2=0x7f183410ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:41.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.057+0000 7f183be70700 1 --2- 172.21.15.31:0/2110071823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1834075590 0x7f183410ef30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1824009990 tx=0x7f1824009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.057+0000 7f183be70700 1 -- 172.21.15.31:0/2110071823 shutdown_connections 2024-04-25T13:18:41.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.057+0000 7f183be70700 1 --2- 172.21.15.31:0/2110071823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1834074c70 0x7f1834075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.057+0000 7f183be70700 1 --2- 172.21.15.31:0/2110071823 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f183410f470 0x7f1834111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.057+0000 7f183be70700 1 --2- 172.21.15.31:0/2110071823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1834075590 0x7f183410ef30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.057+0000 7f183be70700 1 -- 172.21.15.31:0/2110071823 >> 172.21.15.31:0/2110071823 conn(0x7f1834079bb0 msgr2=0x7f1834079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:41.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.057+0000 7f183be70700 1 -- 172.21.15.31:0/2110071823 shutdown_connections 2024-04-25T13:18:41.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.057+0000 7f183be70700 1 -- 172.21.15.31:0/2110071823 wait complete. 2024-04-25T13:18:41.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.057+0000 7f183be70700 1 Processor -- start 2024-04-25T13:18:41.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.057+0000 7f183be70700 1 -- start start 2024-04-25T13:18:41.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183be70700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1834074c70 0x7f183419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:41.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183be70700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1834075590 0x7f183419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:41.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183be70700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f183410f470 0x7f183419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:41.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183be70700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1834114e50 con 0x7f183410f470 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183be70700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1834114cd0 con 0x7f1834074c70 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183be70700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1834114fd0 con 0x7f1834075590 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183a40d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f183410f470 0x7f183419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183a40d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f183410f470 0x7f183419fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39690/0 (socket says 172.21.15.31:39690) 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183a40d700 1 -- 172.21.15.31:0/3798305413 learned_addr learned my addr 172.21.15.31:0/3798305413 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183940b700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1834075590 0x7f183419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f1839c0c700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1834074c70 0x7f183419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183a40d700 1 -- 172.21.15.31:0/3798305413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1834075590 msgr2=0x7f183419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183a40d700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1834075590 0x7f183419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183a40d700 1 -- 172.21.15.31:0/3798305413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1834074c70 msgr2=0x7f183419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183a40d700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1834074c70 0x7f183419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183a40d700 1 -- 172.21.15.31:0/3798305413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18341a0310 con 0x7f183410f470 2024-04-25T13:18:41.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f183a40d700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f183410f470 0x7f183419fbd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f182c007ee0 tx=0x7f182c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f182affd700 1 -- 172.21.15.31:0/3798305413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c004bb0 con 0x7f183410f470 2024-04-25T13:18:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f182affd700 1 -- 172.21.15.31:0/3798305413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f182c004d50 con 0x7f183410f470 2024-04-25T13:18:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.058+0000 7f182affd700 1 -- 172.21.15.31:0/3798305413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c005920 con 0x7f183410f470 2024-04-25T13:18:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.059+0000 7f183be70700 1 -- 172.21.15.31:0/3798305413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18341a0600 con 0x7f183410f470 2024-04-25T13:18:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.059+0000 7f183be70700 1 -- 172.21.15.31:0/3798305413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18341a7e80 con 0x7f183410f470 2024-04-25T13:18:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.059+0000 7f183be70700 1 -- 172.21.15.31:0/3798305413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1834069730 con 0x7f183410f470 2024-04-25T13:18:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.061+0000 7f182affd700 1 -- 172.21.15.31:0/3798305413 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f182c01a020 con 0x7f183410f470 2024-04-25T13:18:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.062+0000 7f182affd700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18200a3cd0 0x7f18200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.062+0000 7f1839c0c700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18200a3cd0 0x7f18200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.062+0000 7f182affd700 1 -- 172.21.15.31:0/3798305413 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f182c0d0270 con 0x7f183410f470 2024-04-25T13:18:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.062+0000 7f1839c0c700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18200a3cd0 0x7f18200a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1830007be0 tx=0x7f18300073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.066+0000 7f182affd700 1 -- 172.21.15.31:0/3798305413 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f182c010070 con 0x7f183410f470 2024-04-25T13:18:41.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:40 smithi031 ceph-mon[29457]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:41.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:40 smithi136 ceph-mon[33038]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.275+0000 7f183be70700 1 -- 172.21.15.31:0/3798305413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f183402df80 con 0x7f183410f470 2024-04-25T13:18:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.275+0000 7f182affd700 1 -- 172.21.15.31:0/3798305413 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f182c09cad0 con 0x7f183410f470 2024-04-25T13:18:41.276 INFO:teuthology.orchestra.run.smithi031.stdout:206158430260 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.277+0000 7f183be70700 1 -- 172.21.15.31:0/3798305413 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18200a3cd0 msgr2=0x7f18200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.277+0000 7f183be70700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18200a3cd0 0x7f18200a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1830007be0 tx=0x7f18300073d0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.277+0000 7f183be70700 1 -- 172.21.15.31:0/3798305413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f183410f470 msgr2=0x7f183419fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.277+0000 7f183be70700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f183410f470 0x7f183419fbd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f182c007ee0 tx=0x7f182c00d3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.278+0000 7f183be70700 1 -- 172.21.15.31:0/3798305413 shutdown_connections 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.278+0000 7f183be70700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f183410f470 0x7f183419fbd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.278+0000 7f183be70700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1834074c70 0x7f183419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.278+0000 7f183be70700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f18200a3cd0 0x7f18200a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.278+0000 7f183be70700 1 --2- 172.21.15.31:0/3798305413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1834075590 0x7f183419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.278+0000 7f183be70700 1 -- 172.21.15.31:0/3798305413 >> 172.21.15.31:0/3798305413 conn(0x7f1834079bb0 msgr2=0x7f18340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.278+0000 7f183be70700 1 -- 172.21.15.31:0/3798305413 shutdown_connections 2024-04-25T13:18:41.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:41.278+0000 7f183be70700 1 -- 172.21.15.31:0/3798305413 wait complete. 2024-04-25T13:18:41.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:40 smithi066 ceph-mon[33137]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:41.504 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574922 for osd.1 2024-04-25T13:18:41.504 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:42.109 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:41 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3798305413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:18:42.109 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:41 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.109 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:41 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.109 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:41 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.109 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:41 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.138+0000 7fd5eb880700 1 -- 172.21.15.31:0/1707568860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e4102a90 msgr2=0x7fd5e4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.138+0000 7fd5eb880700 1 --2- 172.21.15.31:0/1707568860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e4102a90 0x7fd5e4102f10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009990 tx=0x7fd5e0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.138+0000 7fd5eb880700 1 -- 172.21.15.31:0/1707568860 shutdown_connections 2024-04-25T13:18:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.138+0000 7fd5eb880700 1 --2- 172.21.15.31:0/1707568860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4108a50 0x7fd5e4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.138+0000 7fd5eb880700 1 --2- 172.21.15.31:0/1707568860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e4103450 0x7fd5e410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.138+0000 7fd5eb880700 1 --2- 172.21.15.31:0/1707568860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e4102a90 0x7fd5e4102f10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.138+0000 7fd5eb880700 1 -- 172.21.15.31:0/1707568860 >> 172.21.15.31:0/1707568860 conn(0x7fd5e4079ba0 msgr2=0x7fd5e4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.138+0000 7fd5eb880700 1 -- 172.21.15.31:0/1707568860 shutdown_connections 2024-04-25T13:18:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.138+0000 7fd5eb880700 1 -- 172.21.15.31:0/1707568860 wait complete. 2024-04-25T13:18:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5eb880700 1 Processor -- start 2024-04-25T13:18:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5eb880700 1 -- start start 2024-04-25T13:18:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5eb880700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e4102a90 0x7fd5e4199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5eb880700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4103450 0x7fd5e4199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5eb880700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e4108a50 0x7fd5e419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5eb880700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4112c00 con 0x7fd5e4108a50 2024-04-25T13:18:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5eb880700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4112a80 con 0x7fd5e4102a90 2024-04-25T13:18:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5eb880700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4112d80 con 0x7fd5e4103450 2024-04-25T13:18:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5e9e1d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e4108a50 0x7fd5e419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5e9e1d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e4108a50 0x7fd5e419da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40266/0 (socket says 172.21.15.31:40266) 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5e9e1d700 1 -- 172.21.15.31:0/78691799 learned_addr learned my addr 172.21.15.31:0/78691799 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5e961c700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e4102a90 0x7fd5e4199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.139+0000 7fd5e8e1b700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4103450 0x7fd5e4199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5e9e1d700 1 -- 172.21.15.31:0/78691799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4103450 msgr2=0x7fd5e4199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5e9e1d700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4103450 0x7fd5e4199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5e9e1d700 1 -- 172.21.15.31:0/78691799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e4102a90 msgr2=0x7fd5e4199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5e9e1d700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e4102a90 0x7fd5e4199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5e9e1d700 1 -- 172.21.15.31:0/78691799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e419e160 con 0x7fd5e4108a50 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5e961c700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e4102a90 0x7fd5e4199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5e9e1d700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e4108a50 0x7fd5e419da20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc007ee0 tx=0x7fd5dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5da7fc700 1 -- 172.21.15.31:0/78691799 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc004bb0 con 0x7fd5e4108a50 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5da7fc700 1 -- 172.21.15.31:0/78691799 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5dc004d50 con 0x7fd5e4108a50 2024-04-25T13:18:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5da7fc700 1 -- 172.21.15.31:0/78691799 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc005920 con 0x7fd5e4108a50 2024-04-25T13:18:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5eb880700 1 -- 172.21.15.31:0/78691799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e419e450 con 0x7fd5e4108a50 2024-04-25T13:18:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.140+0000 7fd5eb880700 1 -- 172.21.15.31:0/78691799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41a5cd0 con 0x7fd5e4108a50 2024-04-25T13:18:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.141+0000 7fd5eb880700 1 -- 172.21.15.31:0/78691799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e4069730 con 0x7fd5e4108a50 2024-04-25T13:18:42.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.142+0000 7fd5da7fc700 1 -- 172.21.15.31:0/78691799 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd5dc01a020 con 0x7fd5e4108a50 2024-04-25T13:18:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.143+0000 7fd5da7fc700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5d00a3d50 0x7fd5d00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.143+0000 7fd5e961c700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5d00a3d50 0x7fd5d00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.143+0000 7fd5da7fc700 1 -- 172.21.15.31:0/78691799 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fd5dc0d0290 con 0x7fd5e4108a50 2024-04-25T13:18:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.143+0000 7fd5e961c700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5d00a3d50 0x7fd5d00a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40103d0 tx=0x7fd5d400b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:42.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.151+0000 7fd5da7fc700 1 -- 172.21.15.31:0/78691799 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5dc010070 con 0x7fd5e4108a50 2024-04-25T13:18:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:41 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3798305413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:18:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:41 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:41 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:41 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:41 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.369+0000 7fd5eb880700 1 -- 172.21.15.31:0/78691799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd5e402df80 con 0x7fd5e4108a50 2024-04-25T13:18:42.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.369+0000 7fd5da7fc700 1 -- 172.21.15.31:0/78691799 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd5dc09caf0 con 0x7fd5e4108a50 2024-04-25T13:18:42.371 INFO:teuthology.orchestra.run.smithi031.stdout:227633266733 2024-04-25T13:18:42.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 -- 172.21.15.31:0/78691799 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5d00a3d50 msgr2=0x7fd5d00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:42.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5d00a3d50 0x7fd5d00a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40103d0 tx=0x7fd5d400b3c0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 -- 172.21.15.31:0/78691799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e4108a50 msgr2=0x7fd5e419da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:42.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e4108a50 0x7fd5e419da20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc007ee0 tx=0x7fd5dc00d3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 -- 172.21.15.31:0/78691799 shutdown_connections 2024-04-25T13:18:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e4108a50 0x7fd5e419da20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5e4102a90 0x7fd5e4199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd5d00a3d50 0x7fd5d00a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 --2- 172.21.15.31:0/78691799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4103450 0x7fd5e4199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 -- 172.21.15.31:0/78691799 >> 172.21.15.31:0/78691799 conn(0x7fd5e4079ba0 msgr2=0x7fd5e41018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 -- 172.21.15.31:0/78691799 shutdown_connections 2024-04-25T13:18:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.372+0000 7fd5eb880700 1 -- 172.21.15.31:0/78691799 wait complete. 2024-04-25T13:18:42.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:41 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3798305413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:18:42.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:41 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:41 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:41 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:41 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:42.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.937+0000 7f906bd0e700 1 -- 172.21.15.31:0/3724651763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064107990 msgr2=0x7f9064107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:42.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.937+0000 7f906bd0e700 1 --2- 172.21.15.31:0/3724651763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064107990 0x7f9064107d70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9060009990 tx=0x7f9060009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.937+0000 7f906bd0e700 1 -- 172.21.15.31:0/3724651763 shutdown_connections 2024-04-25T13:18:42.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.937+0000 7f906bd0e700 1 --2- 172.21.15.31:0/3724651763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9064102300 0x7f906410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.937+0000 7f906bd0e700 1 --2- 172.21.15.31:0/3724651763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064107990 0x7f9064107d70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.937+0000 7f906bd0e700 1 --2- 172.21.15.31:0/3724651763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064101940 0x7f9064101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.937+0000 7f906bd0e700 1 -- 172.21.15.31:0/3724651763 >> 172.21.15.31:0/3724651763 conn(0x7f90640788f0 msgr2=0x7f9064078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:42.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.938+0000 7f906bd0e700 1 -- 172.21.15.31:0/3724651763 shutdown_connections 2024-04-25T13:18:42.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.938+0000 7f906bd0e700 1 -- 172.21.15.31:0/3724651763 wait complete. 2024-04-25T13:18:42.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.938+0000 7f906bd0e700 1 Processor -- start 2024-04-25T13:18:42.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.938+0000 7f906bd0e700 1 -- start start 2024-04-25T13:18:42.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.938+0000 7f906bd0e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064101940 0x7f9064199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:42.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.938+0000 7f906bd0e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064102300 0x7f90641997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.938+0000 7f906bd0e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9064107990 0x7f906419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.938+0000 7f906bd0e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9064111c20 con 0x7f9064101940 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.938+0000 7f906bd0e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9064111aa0 con 0x7f9064107990 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.938+0000 7f906bd0e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9064111da0 con 0x7f9064102300 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f9069aaa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064101940 0x7f9064199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f9069aaa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064101940 0x7f9064199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40286/0 (socket says 172.21.15.31:40286) 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f9069aaa700 1 -- 172.21.15.31:0/4154742832 learned_addr learned my addr 172.21.15.31:0/4154742832 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f90692a9700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064102300 0x7f90641997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f9069aaa700 1 -- 172.21.15.31:0/4154742832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064102300 msgr2=0x7f90641997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f9069aaa700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064102300 0x7f90641997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f9069aaa700 1 -- 172.21.15.31:0/4154742832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9064107990 msgr2=0x7f906419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f906a2ab700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9064107990 0x7f906419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f9069aaa700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9064107990 0x7f906419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f9069aaa700 1 -- 172.21.15.31:0/4154742832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906419e2a0 con 0x7f9064101940 2024-04-25T13:18:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f906a2ab700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9064107990 0x7f906419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:18:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f9069aaa700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064101940 0x7f9064199290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9060009f90 tx=0x7f9060004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f905affd700 1 -- 172.21.15.31:0/4154742832 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060026070 con 0x7f9064101940 2024-04-25T13:18:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f906bd0e700 1 -- 172.21.15.31:0/4154742832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906419e530 con 0x7f9064101940 2024-04-25T13:18:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f905affd700 1 -- 172.21.15.31:0/4154742832 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90600040b0 con 0x7f9064101940 2024-04-25T13:18:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f905affd700 1 -- 172.21.15.31:0/4154742832 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060005290 con 0x7f9064101940 2024-04-25T13:18:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.939+0000 7f906bd0e700 1 -- 172.21.15.31:0/4154742832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90641a5e10 con 0x7f9064101940 2024-04-25T13:18:42.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.940+0000 7f906bd0e700 1 -- 172.21.15.31:0/4154742832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9064069730 con 0x7f9064101940 2024-04-25T13:18:42.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.942+0000 7f905affd700 1 -- 172.21.15.31:0/4154742832 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9060029070 con 0x7f9064101940 2024-04-25T13:18:42.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.943+0000 7f905affd700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90500a3cd0 0x7f90500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:42.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.943+0000 7f90692a9700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90500a3cd0 0x7f90500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:42.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.943+0000 7f905affd700 1 -- 172.21.15.31:0/4154742832 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f90600d14d0 con 0x7f9064101940 2024-04-25T13:18:42.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.943+0000 7f90692a9700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90500a3cd0 0x7f90500a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9054006fd0 tx=0x7f9054008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:42.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:42.946+0000 7f905affd700 1 -- 172.21.15.31:0/4154742832 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9060030020 con 0x7f9064101940 2024-04-25T13:18:43.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:42 smithi031 ceph-mon[29457]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-25T13:18:43.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:42 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/78691799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:18:43.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.163+0000 7f906bd0e700 1 -- 172.21.15.31:0/4154742832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f90641a5fe0 con 0x7f9064101940 2024-04-25T13:18:43.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.163+0000 7f905affd700 1 -- 172.21.15.31:0/4154742832 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f9060013070 con 0x7f9064101940 2024-04-25T13:18:43.165 INFO:teuthology.orchestra.run.smithi031.stdout:322122547229 2024-04-25T13:18:43.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 -- 172.21.15.31:0/4154742832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90500a3cd0 msgr2=0x7f90500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90500a3cd0 0x7f90500a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9054006fd0 tx=0x7f9054008040 comp rx=0 tx=0).stop 2024-04-25T13:18:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 -- 172.21.15.31:0/4154742832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064101940 msgr2=0x7f9064199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064101940 0x7f9064199290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9060009f90 tx=0x7f9060004630 comp rx=0 tx=0).stop 2024-04-25T13:18:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 -- 172.21.15.31:0/4154742832 shutdown_connections 2024-04-25T13:18:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9064107990 0x7f906419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9064101940 0x7f9064199290 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f90500a3cd0 0x7f90500a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 --2- 172.21.15.31:0/4154742832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9064102300 0x7f90641997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 -- 172.21.15.31:0/4154742832 >> 172.21.15.31:0/4154742832 conn(0x7f90640788f0 msgr2=0x7f906406a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 -- 172.21.15.31:0/4154742832 shutdown_connections 2024-04-25T13:18:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.165+0000 7f906bd0e700 1 -- 172.21.15.31:0/4154742832 wait complete. 2024-04-25T13:18:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:42 smithi136 ceph-mon[33038]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-25T13:18:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:42 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/78691799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:18:43.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.338+0000 7f37c7dfd700 1 -- 172.21.15.31:0/1588867596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0108a70 msgr2=0x7f37c0108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:43.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.338+0000 7f37c7dfd700 1 --2- 172.21.15.31:0/1588867596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0108a70 0x7f37c0108e50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009990 tx=0x7f37b0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.338+0000 7f37c7dfd700 1 -- 172.21.15.31:0/1588867596 shutdown_connections 2024-04-25T13:18:43.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.338+0000 7f37c7dfd700 1 --2- 172.21.15.31:0/1588867596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c0103480 0x7f37c010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.338+0000 7f37c7dfd700 1 --2- 172.21.15.31:0/1588867596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0108a70 0x7f37c0108e50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.338+0000 7f37c7dfd700 1 --2- 172.21.15.31:0/1588867596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37c0102ac0 0x7f37c0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.338+0000 7f37c7dfd700 1 -- 172.21.15.31:0/1588867596 >> 172.21.15.31:0/1588867596 conn(0x7f37c0079bb0 msgr2=0x7f37c0079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.338+0000 7f37c7dfd700 1 -- 172.21.15.31:0/1588867596 shutdown_connections 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.338+0000 7f37c7dfd700 1 -- 172.21.15.31:0/1588867596 wait complete. 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c7dfd700 1 Processor -- start 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c7dfd700 1 -- start start 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c7dfd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0102ac0 0x7f37c0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c7dfd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c0103480 0x7f37c0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c7dfd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37c0108a70 0x7f37c019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c7dfd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37c0112d00 con 0x7f37c0102ac0 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c7dfd700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37c0112b80 con 0x7f37c0103480 2024-04-25T13:18:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c7dfd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f37c0112e80 con 0x7f37c0108a70 2024-04-25T13:18:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c5b99700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0102ac0 0x7f37c0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c5b99700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0102ac0 0x7f37c0199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40292/0 (socket says 172.21.15.31:40292) 2024-04-25T13:18:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c5b99700 1 -- 172.21.15.31:0/2624983889 learned_addr learned my addr 172.21.15.31:0/2624983889 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c639a700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37c0108a70 0x7f37c019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.339+0000 7f37c5398700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c0103480 0x7f37c0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:43.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37c5b99700 1 -- 172.21.15.31:0/2624983889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37c0108a70 msgr2=0x7f37c019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:43.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37c5b99700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37c0108a70 0x7f37c019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37c5b99700 1 -- 172.21.15.31:0/2624983889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c0103480 msgr2=0x7f37c0199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:43.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37c5b99700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c0103480 0x7f37c0199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37c5b99700 1 -- 172.21.15.31:0/2624983889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c019e340 con 0x7f37c0102ac0 2024-04-25T13:18:43.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37c5398700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c0103480 0x7f37c0199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:18:43.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37c5b99700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0102ac0 0x7f37c0199330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f37b0000c00 tx=0x7f37b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:43.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37b6ffd700 1 -- 172.21.15.31:0/2624983889 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0027070 con 0x7f37c0102ac0 2024-04-25T13:18:43.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37b6ffd700 1 -- 172.21.15.31:0/2624983889 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b00040e0 con 0x7f37c0102ac0 2024-04-25T13:18:43.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37c7dfd700 1 -- 172.21.15.31:0/2624983889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c019e5d0 con 0x7f37c0102ac0 2024-04-25T13:18:43.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37b6ffd700 1 -- 172.21.15.31:0/2624983889 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b000f460 con 0x7f37c0102ac0 2024-04-25T13:18:43.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.340+0000 7f37c7dfd700 1 -- 172.21.15.31:0/2624983889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c01a5eb0 con 0x7f37c0102ac0 2024-04-25T13:18:43.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.341+0000 7f37c7dfd700 1 -- 172.21.15.31:0/2624983889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c0069730 con 0x7f37c0102ac0 2024-04-25T13:18:43.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.343+0000 7f37b6ffd700 1 -- 172.21.15.31:0/2624983889 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f37b002a070 con 0x7f37c0102ac0 2024-04-25T13:18:43.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.343+0000 7f37b6ffd700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f37ac0a3d50 0x7f37ac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:43.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.343+0000 7f37c5398700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f37ac0a3d50 0x7f37ac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:43.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.344+0000 7f37b6ffd700 1 -- 172.21.15.31:0/2624983889 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f37b00d1900 con 0x7f37c0102ac0 2024-04-25T13:18:43.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.344+0000 7f37c5398700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f37ac0a3d50 0x7f37ac0a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37c007a2e0 tx=0x7f37bc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:43.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.347+0000 7f37b6ffd700 1 -- 172.21.15.31:0/2624983889 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37b0031070 con 0x7f37c0102ac0 2024-04-25T13:18:43.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:42 smithi066 ceph-mon[33137]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-25T13:18:43.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:42 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/78691799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:18:43.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.557+0000 7f37c7dfd700 1 -- 172.21.15.31:0/2624983889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f37c002df80 con 0x7f37c0102ac0 2024-04-25T13:18:43.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.557+0000 7f37b6ffd700 1 -- 172.21.15.31:0/2624983889 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f37b0013070 con 0x7f37c0102ac0 2024-04-25T13:18:43.559 INFO:teuthology.orchestra.run.smithi031.stdout:261993005097 2024-04-25T13:18:43.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.559+0000 7f37c7dfd700 1 -- 172.21.15.31:0/2624983889 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f37ac0a3d50 msgr2=0x7f37ac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.559+0000 7f37c7dfd700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f37ac0a3d50 0x7f37ac0a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37c007a2e0 tx=0x7f37bc008040 comp rx=0 tx=0).stop 2024-04-25T13:18:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.559+0000 7f37c7dfd700 1 -- 172.21.15.31:0/2624983889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0102ac0 msgr2=0x7f37c0199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.559+0000 7f37c7dfd700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0102ac0 0x7f37c0199330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f37b0000c00 tx=0x7f37b00047b0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.560+0000 7f37c7dfd700 1 -- 172.21.15.31:0/2624983889 shutdown_connections 2024-04-25T13:18:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.560+0000 7f37c7dfd700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f37ac0a3d50 0x7f37ac0a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.560+0000 7f37c7dfd700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37c0108a70 0x7f37c019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.560+0000 7f37c7dfd700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0102ac0 0x7f37c0199330 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.560+0000 7f37c7dfd700 1 --2- 172.21.15.31:0/2624983889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c0103480 0x7f37c0199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.560+0000 7f37c7dfd700 1 -- 172.21.15.31:0/2624983889 >> 172.21.15.31:0/2624983889 conn(0x7f37c0079bb0 msgr2=0x7f37c0101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.560+0000 7f37c7dfd700 1 -- 172.21.15.31:0/2624983889 shutdown_connections 2024-04-25T13:18:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:43.560+0000 7f37c7dfd700 1 -- 172.21.15.31:0/2624983889 wait complete. 2024-04-25T13:18:43.933 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430256 got 206158430260 for osd.6 2024-04-25T13:18:43.933 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:44.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:43 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/4154742832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:18:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:43 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2624983889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:18:44.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:43 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/4154742832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:18:44.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:43 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2624983889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:18:44.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:43 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/4154742832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:18:44.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:43 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2624983889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:18:45.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:44 smithi031 ceph-mon[29457]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-25T13:18:45.309 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:44 smithi066 ceph-mon[33137]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-25T13:18:45.450 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266728 got 227633266733 for osd.7 2024-04-25T13:18:45.450 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:45.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:44 smithi136 ceph-mon[33038]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-25T13:18:45.871 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117055 got 115964117058 for osd.3 2024-04-25T13:18:45.871 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:47.197 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:46 smithi031 ceph-mon[29457]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:47.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:46 smithi066 ceph-mon[33137]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:47.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:46 smithi136 ceph-mon[33038]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:47.545 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005093 got 261993005097 for osd.8 2024-04-25T13:18:47.546 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:48.155 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547223 got 322122547229 for osd.10 2024-04-25T13:18:48.156 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.594+0000 7f45257cd700 1 -- 172.21.15.31:0/3110018156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4520108a60 msgr2=0x7f4520108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.594+0000 7f45257cd700 1 --2- 172.21.15.31:0/3110018156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4520108a60 0x7f4520108e40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f451000b0a0 tx=0x7f451000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.594+0000 7f45257cd700 1 -- 172.21.15.31:0/3110018156 shutdown_connections 2024-04-25T13:18:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.594+0000 7f45257cd700 1 --2- 172.21.15.31:0/3110018156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f452006bbb0 0x7f452010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.594+0000 7f45257cd700 1 --2- 172.21.15.31:0/3110018156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4520108a60 0x7f4520108e40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.594+0000 7f45257cd700 1 --2- 172.21.15.31:0/3110018156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f452006b1f0 0x7f452006b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.594+0000 7f45257cd700 1 -- 172.21.15.31:0/3110018156 >> 172.21.15.31:0/3110018156 conn(0x7f45200789e0 msgr2=0x7f4520078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.594+0000 7f45257cd700 1 -- 172.21.15.31:0/3110018156 shutdown_connections 2024-04-25T13:18:48.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.594+0000 7f45257cd700 1 -- 172.21.15.31:0/3110018156 wait complete. 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.594+0000 7f45257cd700 1 Processor -- start 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.594+0000 7f45257cd700 1 -- start start 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f45257cd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f452006b1f0 0x7f452019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f45257cd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f452006bbb0 0x7f452019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f45257cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4520108a60 0x7f452019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f45257cd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4520112d00 con 0x7f4520108a60 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f45257cd700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4520112b80 con 0x7f452006b1f0 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f45257cd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4520112e80 con 0x7f452006bbb0 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4520108a60 0x7f452019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4520108a60 0x7f452019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40308/0 (socket says 172.21.15.31:40308) 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451f7fe700 1 -- 172.21.15.31:0/2760745853 learned_addr learned my addr 172.21.15.31:0/2760745853 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451effd700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f452006b1f0 0x7f452019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451e7fc700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f452006bbb0 0x7f452019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:48.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451f7fe700 1 -- 172.21.15.31:0/2760745853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f452006bbb0 msgr2=0x7f452019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:48.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451f7fe700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f452006bbb0 0x7f452019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451f7fe700 1 -- 172.21.15.31:0/2760745853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f452006b1f0 msgr2=0x7f452019b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:48.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451f7fe700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f452006b1f0 0x7f452019b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451f7fe700 1 -- 172.21.15.31:0/2760745853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45201a04e0 con 0x7f4520108a60 2024-04-25T13:18:48.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451e7fc700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f452006bbb0 0x7f452019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:18:48.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.595+0000 7f451f7fe700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4520108a60 0x7f452019fda0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f451800b550 tx=0x7f451800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:48.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.596+0000 7f4507fff700 1 -- 172.21.15.31:0/2760745853 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518015020 con 0x7f4520108a60 2024-04-25T13:18:48.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.596+0000 7f4507fff700 1 -- 172.21.15.31:0/2760745853 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4518011ce0 con 0x7f4520108a60 2024-04-25T13:18:48.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.596+0000 7f4507fff700 1 -- 172.21.15.31:0/2760745853 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518010dd0 con 0x7f4520108a60 2024-04-25T13:18:48.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.596+0000 7f45257cd700 1 -- 172.21.15.31:0/2760745853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45201a07d0 con 0x7f4520108a60 2024-04-25T13:18:48.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.596+0000 7f45257cd700 1 -- 172.21.15.31:0/2760745853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45201a8050 con 0x7f4520108a60 2024-04-25T13:18:48.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.597+0000 7f45257cd700 1 -- 172.21.15.31:0/2760745853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4520069730 con 0x7f4520108a60 2024-04-25T13:18:48.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.598+0000 7f4507fff700 1 -- 172.21.15.31:0/2760745853 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4518018020 con 0x7f4520108a60 2024-04-25T13:18:48.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.599+0000 7f4507fff700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f45080a3d50 0x7f45080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.599+0000 7f451effd700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f45080a3d50 0x7f45080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.599+0000 7f4507fff700 1 -- 172.21.15.31:0/2760745853 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f45180d00c0 con 0x7f4520108a60 2024-04-25T13:18:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.599+0000 7f451effd700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f45080a3d50 0x7f45080a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4510009fd0 tx=0x7f451000b7b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.603+0000 7f4507fff700 1 -- 172.21.15.31:0/2760745853 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f451801f020 con 0x7f4520108a60 2024-04-25T13:18:48.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.777+0000 7ff011c1c700 1 -- 172.21.15.31:0/162725990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c06a750 msgr2=0x7ff00c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:48.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.777+0000 7ff011c1c700 1 --2- 172.21.15.31:0/162725990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c06a750 0x7ff00c06abb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff000009990 tx=0x7ff000009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.777+0000 7ff011c1c700 1 -- 172.21.15.31:0/162725990 shutdown_connections 2024-04-25T13:18:48.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.777+0000 7ff011c1c700 1 --2- 172.21.15.31:0/162725990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff00c108a30 0x7ff00c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.777+0000 7ff011c1c700 1 --2- 172.21.15.31:0/162725990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff00c06b0f0 0x7ff00c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.777+0000 7ff011c1c700 1 --2- 172.21.15.31:0/162725990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c06a750 0x7ff00c06abb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.777+0000 7ff011c1c700 1 -- 172.21.15.31:0/162725990 >> 172.21.15.31:0/162725990 conn(0x7ff00c0788f0 msgr2=0x7ff00c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:48.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.777+0000 7ff011c1c700 1 -- 172.21.15.31:0/162725990 shutdown_connections 2024-04-25T13:18:48.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.777+0000 7ff011c1c700 1 -- 172.21.15.31:0/162725990 wait complete. 2024-04-25T13:18:48.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.778+0000 7ff011c1c700 1 Processor -- start 2024-04-25T13:18:48.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.778+0000 7ff011c1c700 1 -- start start 2024-04-25T13:18:48.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.778+0000 7ff011c1c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c06a750 0x7ff00c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:48.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.778+0000 7ff011c1c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff00c06b0f0 0x7ff00c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:48.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.778+0000 7ff011c1c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff00c108a30 0x7ff00c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:48.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.778+0000 7ff011c1c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c112bd0 con 0x7ff00c06a750 2024-04-25T13:18:48.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.778+0000 7ff011c1c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c112a50 con 0x7ff00c06b0f0 2024-04-25T13:18:48.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.778+0000 7ff011c1c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c112d50 con 0x7ff00c108a30 2024-04-25T13:18:48.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff00b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c06a750 0x7ff00c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff00b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c06a750 0x7ff00c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40314/0 (socket says 172.21.15.31:40314) 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff00b7fe700 1 -- 172.21.15.31:0/698853779 learned_addr learned my addr 172.21.15.31:0/698853779 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff00affd700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff00c06b0f0 0x7ff00c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff00bfff700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff00c108a30 0x7ff00c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff00b7fe700 1 -- 172.21.15.31:0/698853779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff00c108a30 msgr2=0x7ff00c19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff00b7fe700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff00c108a30 0x7ff00c19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff00b7fe700 1 -- 172.21.15.31:0/698853779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff00c06b0f0 msgr2=0x7ff00c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff00b7fe700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff00c06b0f0 0x7ff00c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff00b7fe700 1 -- 172.21.15.31:0/698853779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c19e0e0 con 0x7ff00c06a750 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff00b7fe700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c06a750 0x7ff00c199120 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feffc00d980 tx=0x7feffc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff008ff9700 1 -- 172.21.15.31:0/698853779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc017070 con 0x7ff00c06a750 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff011c1c700 1 -- 172.21.15.31:0/698853779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00c19e3d0 con 0x7ff00c06a750 2024-04-25T13:18:48.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.779+0000 7ff008ff9700 1 -- 172.21.15.31:0/698853779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feffc005020 con 0x7ff00c06a750 2024-04-25T13:18:48.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.780+0000 7ff008ff9700 1 -- 172.21.15.31:0/698853779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc00fa10 con 0x7ff00c06a750 2024-04-25T13:18:48.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.780+0000 7ff011c1c700 1 -- 172.21.15.31:0/698853779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00c1a5c50 con 0x7ff00c06a750 2024-04-25T13:18:48.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.781+0000 7ff011c1c700 1 -- 172.21.15.31:0/698853779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00c069730 con 0x7ff00c06a750 2024-04-25T13:18:48.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.782+0000 7ff008ff9700 1 -- 172.21.15.31:0/698853779 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feffc004a30 con 0x7ff00c06a750 2024-04-25T13:18:48.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.782+0000 7ff008ff9700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7feff40a3cd0 0x7feff40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:48.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.782+0000 7ff00affd700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7feff40a3cd0 0x7feff40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:48.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.782+0000 7ff008ff9700 1 -- 172.21.15.31:0/698853779 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7feffc0d03a0 con 0x7ff00c06a750 2024-04-25T13:18:48.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.783+0000 7ff00affd700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7feff40a3cd0 0x7feff40a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff0000098e0 tx=0x7ff000005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:48.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.787+0000 7ff008ff9700 1 -- 172.21.15.31:0/698853779 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feffc020020 con 0x7ff00c06a750 2024-04-25T13:18:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.817+0000 7f45257cd700 1 -- 172.21.15.31:0/2760745853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f452019c830 con 0x7f4520108a60 2024-04-25T13:18:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.818+0000 7f4507fff700 1 -- 172.21.15.31:0/2760745853 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f451809ca30 con 0x7f4520108a60 2024-04-25T13:18:48.819 INFO:teuthology.orchestra.run.smithi031.stdout:150323855423 2024-04-25T13:18:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 -- 172.21.15.31:0/2760745853 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f45080a3d50 msgr2=0x7f45080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f45080a3d50 0x7f45080a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4510009fd0 tx=0x7f451000b7b0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 -- 172.21.15.31:0/2760745853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4520108a60 msgr2=0x7f452019fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4520108a60 0x7f452019fda0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f451800b550 tx=0x7f451800b860 comp rx=0 tx=0).stop 2024-04-25T13:18:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 -- 172.21.15.31:0/2760745853 shutdown_connections 2024-04-25T13:18:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4520108a60 0x7f452019fda0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f452006b1f0 0x7f452019b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f45080a3d50 0x7f45080a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 --2- 172.21.15.31:0/2760745853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f452006bbb0 0x7f452019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:48.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 -- 172.21.15.31:0/2760745853 >> 172.21.15.31:0/2760745853 conn(0x7f45200789e0 msgr2=0x7f452010dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:48.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 -- 172.21.15.31:0/2760745853 shutdown_connections 2024-04-25T13:18:48.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:48.820+0000 7f45257cd700 1 -- 172.21.15.31:0/2760745853 wait complete. 2024-04-25T13:18:49.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.003+0000 7ff011c1c700 1 -- 172.21.15.31:0/698853779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff00c02df80 con 0x7ff00c06a750 2024-04-25T13:18:49.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.004+0000 7ff008ff9700 1 -- 172.21.15.31:0/698853779 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feffc09cc00 con 0x7ff00c06a750 2024-04-25T13:18:49.005 INFO:teuthology.orchestra.run.smithi031.stdout:296352743461 2024-04-25T13:18:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.006+0000 7ff011c1c700 1 -- 172.21.15.31:0/698853779 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7feff40a3cd0 msgr2=0x7feff40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:49.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.006+0000 7ff011c1c700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7feff40a3cd0 0x7feff40a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff0000098e0 tx=0x7ff000005960 comp rx=0 tx=0).stop 2024-04-25T13:18:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.006+0000 7ff011c1c700 1 -- 172.21.15.31:0/698853779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c06a750 msgr2=0x7ff00c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.006+0000 7ff011c1c700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c06a750 0x7ff00c199120 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feffc00d980 tx=0x7feffc00dc90 comp rx=0 tx=0).stop 2024-04-25T13:18:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.006+0000 7ff011c1c700 1 -- 172.21.15.31:0/698853779 shutdown_connections 2024-04-25T13:18:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.006+0000 7ff011c1c700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7feff40a3cd0 0x7feff40a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.006+0000 7ff011c1c700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff00c108a30 0x7ff00c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.006+0000 7ff011c1c700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff00c06a750 0x7ff00c199120 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.006+0000 7ff011c1c700 1 --2- 172.21.15.31:0/698853779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff00c06b0f0 0x7ff00c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.006+0000 7ff011c1c700 1 -- 172.21.15.31:0/698853779 >> 172.21.15.31:0/698853779 conn(0x7ff00c0788f0 msgr2=0x7ff00c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.007+0000 7ff011c1c700 1 -- 172.21.15.31:0/698853779 shutdown_connections 2024-04-25T13:18:49.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.007+0000 7ff011c1c700 1 -- 172.21.15.31:0/698853779 wait complete. 2024-04-25T13:18:49.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:48 smithi031 ceph-mon[29457]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:49.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:48 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2760745853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:18:49.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:48 smithi066 ceph-mon[33137]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:49.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:48 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2760745853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:18:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:48 smithi136 ceph-mon[33038]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:48 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2760745853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:18:49.538 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855418 got 150323855423 for osd.4 2024-04-25T13:18:49.538 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:49.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.857+0000 7f468a861700 1 -- 172.21.15.31:0/3369904803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4684075590 msgr2=0x7f468410ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:49.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.857+0000 7f468a861700 1 --2- 172.21.15.31:0/3369904803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4684075590 0x7f468410ef30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4678009990 tx=0x7f4678009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:49.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.857+0000 7f468a861700 1 -- 172.21.15.31:0/3369904803 shutdown_connections 2024-04-25T13:18:49.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.857+0000 7f468a861700 1 --2- 172.21.15.31:0/3369904803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4684074c70 0x7f4684075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:49.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.857+0000 7f468a861700 1 --2- 172.21.15.31:0/3369904803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f468410f470 0x7f4684111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:49.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.857+0000 7f468a861700 1 --2- 172.21.15.31:0/3369904803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4684075590 0x7f468410ef30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:49.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.857+0000 7f468a861700 1 -- 172.21.15.31:0/3369904803 >> 172.21.15.31:0/3369904803 conn(0x7f4684079bf0 msgr2=0x7f468407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:49.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.857+0000 7f468a861700 1 -- 172.21.15.31:0/3369904803 shutdown_connections 2024-04-25T13:18:49.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.857+0000 7f468a861700 1 -- 172.21.15.31:0/3369904803 wait complete. 2024-04-25T13:18:49.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.858+0000 7f468a861700 1 Processor -- start 2024-04-25T13:18:49.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.858+0000 7f468a861700 1 -- start start 2024-04-25T13:18:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.858+0000 7f468a861700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4684074c70 0x7f468419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.858+0000 7f468a861700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4684075590 0x7f468419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.858+0000 7f468a861700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f468410f470 0x7f468419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.858+0000 7f468a861700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4684114e80 con 0x7f4684075590 2024-04-25T13:18:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.858+0000 7f468a861700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f4684114d00 con 0x7f4684074c70 2024-04-25T13:18:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.858+0000 7f468a861700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4684115000 con 0x7f468410f470 2024-04-25T13:18:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.858+0000 7f46837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4684075590 0x7f468419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f46837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4684075590 0x7f468419b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40336/0 (socket says 172.21.15.31:40336) 2024-04-25T13:18:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f46837fe700 1 -- 172.21.15.31:0/3676566199 learned_addr learned my addr 172.21.15.31:0/3676566199 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f4688dfe700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f468410f470 0x7f468419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f4683fff700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4684074c70 0x7f468419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f46837fe700 1 -- 172.21.15.31:0/3676566199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f468410f470 msgr2=0x7f468419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f46837fe700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f468410f470 0x7f468419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f46837fe700 1 -- 172.21.15.31:0/3676566199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4684074c70 msgr2=0x7f468419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f46837fe700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4684074c70 0x7f468419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f46837fe700 1 -- 172.21.15.31:0/3676566199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46841a03a0 con 0x7f4684075590 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f46837fe700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4684075590 0x7f468419b8d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4678000c00 tx=0x7f4678009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f46817fa700 1 -- 172.21.15.31:0/3676566199 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678027070 con 0x7f4684075590 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f46817fa700 1 -- 172.21.15.31:0/3676566199 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4678022470 con 0x7f4684075590 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f468a861700 1 -- 172.21.15.31:0/3676566199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46841a0630 con 0x7f4684075590 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f46817fa700 1 -- 172.21.15.31:0/3676566199 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678017650 con 0x7f4684075590 2024-04-25T13:18:49.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.859+0000 7f468a861700 1 -- 172.21.15.31:0/3676566199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46841a7f10 con 0x7f4684075590 2024-04-25T13:18:49.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.860+0000 7f468a861700 1 -- 172.21.15.31:0/3676566199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46840763b0 con 0x7f4684075590 2024-04-25T13:18:49.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.862+0000 7f46817fa700 1 -- 172.21.15.31:0/3676566199 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f467802a070 con 0x7f4684075590 2024-04-25T13:18:49.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.862+0000 7f46817fa700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f466c0a3cd0 0x7f466c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:49.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.862+0000 7f4683fff700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f466c0a3cd0 0x7f466c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:49.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.862+0000 7f46817fa700 1 -- 172.21.15.31:0/3676566199 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f46780d17b0 con 0x7f4684075590 2024-04-25T13:18:49.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.863+0000 7f4683fff700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f466c0a3cd0 0x7f466c0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4674005950 tx=0x7f46740058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:49.866+0000 7f46817fa700 1 -- 172.21.15.31:0/3676566199 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4678013070 con 0x7f4684075590 2024-04-25T13:18:50.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.077+0000 7f468a861700 1 -- 172.21.15.31:0/3676566199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f468402df80 con 0x7f4684075590 2024-04-25T13:18:50.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.078+0000 7f46817fa700 1 -- 172.21.15.31:0/3676566199 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f467809e0a0 con 0x7f4684075590 2024-04-25T13:18:50.079 INFO:teuthology.orchestra.run.smithi031.stdout:356482285595 2024-04-25T13:18:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 -- 172.21.15.31:0/3676566199 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f466c0a3cd0 msgr2=0x7f466c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f466c0a3cd0 0x7f466c0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4674005950 tx=0x7f46740058e0 comp rx=0 tx=0).stop 2024-04-25T13:18:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 -- 172.21.15.31:0/3676566199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4684075590 msgr2=0x7f468419b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4684075590 0x7f468419b8d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4678000c00 tx=0x7f4678009580 comp rx=0 tx=0).stop 2024-04-25T13:18:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 -- 172.21.15.31:0/3676566199 shutdown_connections 2024-04-25T13:18:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f466c0a3cd0 0x7f466c0a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:50.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f468410f470 0x7f468419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:50.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4684074c70 0x7f468419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:50.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 --2- 172.21.15.31:0/3676566199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4684075590 0x7f468419b8d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:50.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 -- 172.21.15.31:0/3676566199 >> 172.21.15.31:0/3676566199 conn(0x7f4684079bf0 msgr2=0x7f46840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:50.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 -- 172.21.15.31:0/3676566199 shutdown_connections 2024-04-25T13:18:50.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:50.080+0000 7f468a861700 1 -- 172.21.15.31:0/3676566199 wait complete. 2024-04-25T13:18:50.093 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:49 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/698853779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:18:50.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:49 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/698853779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:18:50.468 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:49 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/698853779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:18:50.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743456 got 296352743461 for osd.9 2024-04-25T13:18:50.533 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:51.099 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:50 smithi031 ceph-mon[29457]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:51.099 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:50 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3676566199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:18:51.132 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285588 got 356482285595 for osd.11 2024-04-25T13:18:51.132 DEBUG:teuthology.parallel:result is None 2024-04-25T13:18:51.132 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T13:18:51.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:18:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:50 smithi066 ceph-mon[33137]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:50 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3676566199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:18:51.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:50 smithi136 ceph-mon[33038]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:51.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:50 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3676566199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:18:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:53.240 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:52 smithi031 ceph-mon[29457]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:52 smithi066 ceph-mon[33137]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:53.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:52 smithi136 ceph-mon[33038]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:53.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.548+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1288406057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b8101400 msgr2=0x7f05b8101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.548+0000 7f05bc8d9700 1 --2- 172.21.15.31:0/1288406057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b8101400 0x7f05b8101880 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009990 tx=0x7f05ac009ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.548+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1288406057 shutdown_connections 2024-04-25T13:18:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.548+0000 7f05bc8d9700 1 --2- 172.21.15.31:0/1288406057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05b8101dc0 0x7f05b810f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.548+0000 7f05bc8d9700 1 --2- 172.21.15.31:0/1288406057 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b8107b20 0x7f05b8107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.548+0000 7f05bc8d9700 1 --2- 172.21.15.31:0/1288406057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b8101400 0x7f05b8101880 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.548+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1288406057 >> 172.21.15.31:0/1288406057 conn(0x7f05b8076c60 msgr2=0x7f05b8077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.548+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1288406057 shutdown_connections 2024-04-25T13:18:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.548+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1288406057 wait complete. 2024-04-25T13:18:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.548+0000 7f05bc8d9700 1 Processor -- start 2024-04-25T13:18:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.548+0000 7f05bc8d9700 1 -- start start 2024-04-25T13:18:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05bc8d9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05b8101400 0x7f05b8199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05bc8d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b8101dc0 0x7f05b81995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05bc8d9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b8107b20 0x7f05b819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05bc8d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8112c70 con 0x7f05b8101dc0 2024-04-25T13:18:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05bc8d9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8112af0 con 0x7f05b8107b20 2024-04-25T13:18:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05bc8d9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8112df0 con 0x7f05b8101400 2024-04-25T13:18:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05b5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b8101dc0 0x7f05b81995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05b5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b8101dc0 0x7f05b81995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58128/0 (socket says 172.21.15.31:58128) 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05b5d9b700 1 -- 172.21.15.31:0/1134472860 learned_addr learned my addr 172.21.15.31:0/1134472860 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05b6d9d700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b8107b20 0x7f05b819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05b659c700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05b8101400 0x7f05b8199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05b5d9b700 1 -- 172.21.15.31:0/1134472860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05b8101400 msgr2=0x7f05b8199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05b5d9b700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05b8101400 0x7f05b8199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05b5d9b700 1 -- 172.21.15.31:0/1134472860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b8107b20 msgr2=0x7f05b819d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05b5d9b700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b8107b20 0x7f05b819d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05b5d9b700 1 -- 172.21.15.31:0/1134472860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b819e020 con 0x7f05b8101dc0 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.549+0000 7f05b5d9b700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b8101dc0 0x7f05b81995a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0057c0 tx=0x7f05ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.550+0000 7f05a77fe700 1 -- 172.21.15.31:0/1134472860 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac027070 con 0x7f05b8101dc0 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.550+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1134472860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b819e2b0 con 0x7f05b8101dc0 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.550+0000 7f05a77fe700 1 -- 172.21.15.31:0/1134472860 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05ac003f20 con 0x7f05b8101dc0 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.550+0000 7f05a77fe700 1 -- 172.21.15.31:0/1134472860 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac022b30 con 0x7f05b8101dc0 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.550+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1134472860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b81a5b90 con 0x7f05b8101dc0 2024-04-25T13:18:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.551+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1134472860 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b8069730 con 0x7f05b8101dc0 2024-04-25T13:18:53.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.552+0000 7f05a77fe700 1 -- 172.21.15.31:0/1134472860 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f05ac02a070 con 0x7f05b8101dc0 2024-04-25T13:18:53.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.552+0000 7f05a77fe700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f05a00a3cd0 0x7f05a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:53.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.552+0000 7f05b659c700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f05a00a3cd0 0x7f05a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:53.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.553+0000 7f05a77fe700 1 -- 172.21.15.31:0/1134472860 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f05ac0d1470 con 0x7f05b8101dc0 2024-04-25T13:18:53.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.553+0000 7f05b659c700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f05a00a3cd0 0x7f05a00a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f05b80781e0 tx=0x7f05a800a3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:53.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.557+0000 7f05a77fe700 1 -- 172.21.15.31:0/1134472860 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05ac05eff0 con 0x7f05b8101dc0 2024-04-25T13:18:53.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.784+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1134472860 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05b819a370 con 0x7f05a00a3cd0 2024-04-25T13:18:53.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.791+0000 7f05a77fe700 1 -- 172.21.15.31:0/1134472860 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258632 (secure 0 0 0) 0x7f05b819a370 con 0x7f05a00a3cd0 2024-04-25T13:18:53.796 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:18:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1134472860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f05a00a3cd0 msgr2=0x7f05a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f05a00a3cd0 0x7f05a00a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f05b80781e0 tx=0x7f05a800a3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1134472860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b8101dc0 msgr2=0x7f05b81995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b8101dc0 0x7f05b81995a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0057c0 tx=0x7f05ac0057f0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1134472860 shutdown_connections 2024-04-25T13:18:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f05a00a3cd0 0x7f05a00a6190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05b8101400 0x7f05b8199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05b8107b20 0x7f05b819d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 --2- 172.21.15.31:0/1134472860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05b8101dc0 0x7f05b81995a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:53.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1134472860 >> 172.21.15.31:0/1134472860 conn(0x7f05b8076c60 msgr2=0x7f05b80ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:53.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1134472860 shutdown_connections 2024-04-25T13:18:53.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:53.795+0000 7f05bc8d9700 1 -- 172.21.15.31:0/1134472860 wait complete. 2024-04-25T13:18:53.799 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T13:18:54.355 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-04-25T13:18:51.818273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652446,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":279,"num_read_kb":252,"num_write":258,"num_write_kb":3023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387644,"kb_used_data":13452,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554180,"statfs":{"total":1151940427776,"available":1151543480320,"internally_reserved":0,"allocated":13774848,"data_stored":7587030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003088"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948838+0000","last_change":"2024-04-25T13:17:40.825728+0000","last_active":"2024-04-25T13:18:32.948838+0000","last_peered":"2024-04-25T13:18:32.948838+0000","last_clean":"2024-04-25T13:18:32.948838+0000","last_became_active":"2024-04-25T13:17:40.824933+0000","last_became_peered":"2024-04-25T13:17:40.824933+0000","last_unstale":"2024-04-25T13:18:32.948838+0000","last_undegraded":"2024-04-25T13:18:32.948838+0000","last_fullsized":"2024-04-25T13:18:32.948838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:14.326614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948553+0000","last_change":"2024-04-25T13:17:40.821148+0000","last_active":"2024-04-25T13:18:32.948553+0000","last_peered":"2024-04-25T13:18:32.948553+0000","last_clean":"2024-04-25T13:18:32.948553+0000","last_became_active":"2024-04-25T13:17:40.820244+0000","last_became_peered":"2024-04-25T13:17:40.820244+0000","last_unstale":"2024-04-25T13:18:32.948553+0000","last_undegraded":"2024-04-25T13:18:32.948553+0000","last_fullsized":"2024-04-25T13:18:32.948553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.566793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948485+0000","last_change":"2024-04-25T13:17:40.818729+0000","last_active":"2024-04-25T13:18:32.948485+0000","last_peered":"2024-04-25T13:18:32.948485+0000","last_clean":"2024-04-25T13:18:32.948485+0000","last_became_active":"2024-04-25T13:17:40.818351+0000","last_became_peered":"2024-04-25T13:17:40.818351+0000","last_unstale":"2024-04-25T13:18:32.948485+0000","last_undegraded":"2024-04-25T13:18:32.948485+0000","last_fullsized":"2024-04-25T13:18:32.948485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:42.412891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.289974+0000","last_change":"2024-04-25T13:17:40.820932+0000","last_active":"2024-04-25T13:18:33.289974+0000","last_peered":"2024-04-25T13:18:33.289974+0000","last_clean":"2024-04-25T13:18:33.289974+0000","last_became_active":"2024-04-25T13:17:40.820525+0000","last_became_peered":"2024-04-25T13:17:40.820525+0000","last_unstale":"2024-04-25T13:18:33.289974+0000","last_undegraded":"2024-04-25T13:18:33.289974+0000","last_fullsized":"2024-04-25T13:18:33.289974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:12.077703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949658+0000","last_change":"2024-04-25T13:17:40.829959+0000","last_active":"2024-04-25T13:18:32.949658+0000","last_peered":"2024-04-25T13:18:32.949658+0000","last_clean":"2024-04-25T13:18:32.949658+0000","last_became_active":"2024-04-25T13:17:40.829594+0000","last_became_peered":"2024-04-25T13:17:40.829594+0000","last_unstale":"2024-04-25T13:18:32.949658+0000","last_undegraded":"2024-04-25T13:18:32.949658+0000","last_fullsized":"2024-04-25T13:18:32.949658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:55.719296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332296+0000","last_change":"2024-04-25T13:17:40.817436+0000","last_active":"2024-04-25T13:18:33.332296+0000","last_peered":"2024-04-25T13:18:33.332296+0000","last_clean":"2024-04-25T13:18:33.332296+0000","last_became_active":"2024-04-25T13:17:40.816763+0000","last_became_peered":"2024-04-25T13:17:40.816763+0000","last_unstale":"2024-04-25T13:18:33.332296+0000","last_undegraded":"2024-04-25T13:18:33.332296+0000","last_fullsized":"2024-04-25T13:18:33.332296+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:09.505890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.166779+0000","last_change":"2024-04-25T13:17:40.822968+0000","last_active":"2024-04-25T13:18:33.166779+0000","last_peered":"2024-04-25T13:18:33.166779+0000","last_clean":"2024-04-25T13:18:33.166779+0000","last_became_active":"2024-04-25T13:17:40.822600+0000","last_became_peered":"2024-04-25T13:17:40.822600+0000","last_unstale":"2024-04-25T13:18:33.166779+0000","last_undegraded":"2024-04-25T13:18:33.166779+0000","last_fullsized":"2024-04-25T13:18:33.166779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:08:32.276310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189451+0000","last_change":"2024-04-25T13:17:40.823421+0000","last_active":"2024-04-25T13:18:33.189451+0000","last_peered":"2024-04-25T13:18:33.189451+0000","last_clean":"2024-04-25T13:18:33.189451+0000","last_became_active":"2024-04-25T13:17:40.823054+0000","last_became_peered":"2024-04-25T13:17:40.823054+0000","last_unstale":"2024-04-25T13:18:33.189451+0000","last_undegraded":"2024-04-25T13:18:33.189451+0000","last_fullsized":"2024-04-25T13:18:33.189451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:34.256763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.951026+0000","last_change":"2024-04-25T13:17:40.826105+0000","last_active":"2024-04-25T13:18:32.951026+0000","last_peered":"2024-04-25T13:18:32.951026+0000","last_clean":"2024-04-25T13:18:32.951026+0000","last_became_active":"2024-04-25T13:17:40.825763+0000","last_became_peered":"2024-04-25T13:17:40.825763+0000","last_unstale":"2024-04-25T13:18:32.951026+0000","last_undegraded":"2024-04-25T13:18:32.951026+0000","last_fullsized":"2024-04-25T13:18:32.951026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.058115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332705+0000","last_change":"2024-04-25T13:17:40.817356+0000","last_active":"2024-04-25T13:18:33.332705+0000","last_peered":"2024-04-25T13:18:33.332705+0000","last_clean":"2024-04-25T13:18:33.332705+0000","last_became_active":"2024-04-25T13:17:40.816906+0000","last_became_peered":"2024-04-25T13:17:40.816906+0000","last_unstale":"2024-04-25T13:18:33.332705+0000","last_undegraded":"2024-04-25T13:18:33.332705+0000","last_fullsized":"2024-04-25T13:18:33.332705+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167600+0000","last_change":"2024-04-25T13:17:40.829445+0000","last_active":"2024-04-25T13:18:33.167600+0000","last_peered":"2024-04-25T13:18:33.167600+0000","last_clean":"2024-04-25T13:18:33.167600+0000","last_became_active":"2024-04-25T13:17:40.826709+0000","last_became_peered":"2024-04-25T13:17:40.826709+0000","last_unstale":"2024-04-25T13:18:33.167600+0000","last_undegraded":"2024-04-25T13:18:33.167600+0000","last_fullsized":"2024-04-25T13:18:33.167600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:43.044192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.289827+0000","last_change":"2024-04-25T13:17:40.827486+0000","last_active":"2024-04-25T13:18:33.289827+0000","last_peered":"2024-04-25T13:18:33.289827+0000","last_clean":"2024-04-25T13:18:33.289827+0000","last_became_active":"2024-04-25T13:17:40.827070+0000","last_became_peered":"2024-04-25T13:17:40.827070+0000","last_unstale":"2024-04-25T13:18:33.289827+0000","last_undegraded":"2024-04-25T13:18:33.289827+0000","last_fullsized":"2024-04-25T13:18:33.289827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:39.564567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945336+0000","last_change":"2024-04-25T13:17:40.826645+0000","last_active":"2024-04-25T13:18:32.945336+0000","last_peered":"2024-04-25T13:18:32.945336+0000","last_clean":"2024-04-25T13:18:32.945336+0000","last_became_active":"2024-04-25T13:17:40.826314+0000","last_became_peered":"2024-04-25T13:17:40.826314+0000","last_unstale":"2024-04-25T13:18:32.945336+0000","last_undegraded":"2024-04-25T13:18:32.945336+0000","last_fullsized":"2024-04-25T13:18:32.945336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:59.600451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.947973+0000","last_change":"2024-04-25T13:17:40.814918+0000","last_active":"2024-04-25T13:18:32.947973+0000","last_peered":"2024-04-25T13:18:32.947973+0000","last_clean":"2024-04-25T13:18:32.947973+0000","last_became_active":"2024-04-25T13:17:40.814573+0000","last_became_peered":"2024-04-25T13:17:40.814573+0000","last_unstale":"2024-04-25T13:18:32.947973+0000","last_undegraded":"2024-04-25T13:18:32.947973+0000","last_fullsized":"2024-04-25T13:18:32.947973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:34.554390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.947805+0000","last_change":"2024-04-25T13:17:40.825481+0000","last_active":"2024-04-25T13:18:32.947805+0000","last_peered":"2024-04-25T13:18:32.947805+0000","last_clean":"2024-04-25T13:18:32.947805+0000","last_became_active":"2024-04-25T13:17:40.823726+0000","last_became_peered":"2024-04-25T13:17:40.823726+0000","last_unstale":"2024-04-25T13:18:32.947805+0000","last_undegraded":"2024-04-25T13:18:32.947805+0000","last_fullsized":"2024-04-25T13:18:32.947805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:46.500031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.950347+0000","last_change":"2024-04-25T13:17:40.827561+0000","last_active":"2024-04-25T13:18:32.950347+0000","last_peered":"2024-04-25T13:18:32.950347+0000","last_clean":"2024-04-25T13:18:32.950347+0000","last_became_active":"2024-04-25T13:17:40.827100+0000","last_became_peered":"2024-04-25T13:17:40.827100+0000","last_unstale":"2024-04-25T13:18:32.950347+0000","last_undegraded":"2024-04-25T13:18:32.950347+0000","last_fullsized":"2024-04-25T13:18:32.950347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:41.381469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949046+0000","last_change":"2024-04-25T13:17:40.826017+0000","last_active":"2024-04-25T13:18:32.949046+0000","last_peered":"2024-04-25T13:18:32.949046+0000","last_clean":"2024-04-25T13:18:32.949046+0000","last_became_active":"2024-04-25T13:17:40.825429+0000","last_became_peered":"2024-04-25T13:17:40.825429+0000","last_unstale":"2024-04-25T13:18:32.949046+0000","last_undegraded":"2024-04-25T13:18:32.949046+0000","last_fullsized":"2024-04-25T13:18:32.949046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:58.788260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.333588+0000","last_change":"2024-04-25T13:18:33.333588+0000","last_active":"2024-04-25T13:18:33.333588+0000","last_peered":"2024-04-25T13:18:33.333588+0000","last_clean":"2024-04-25T13:18:33.333588+0000","last_became_active":"2024-04-25T13:18:33.333310+0000","last_became_peered":"2024-04-25T13:18:33.333310+0000","last_unstale":"2024-04-25T13:18:33.333588+0000","last_undegraded":"2024-04-25T13:18:33.333588+0000","last_fullsized":"2024-04-25T13:18:33.333588+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:04.363715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964751+0000","last_change":"2024-04-25T13:17:40.824689+0000","last_active":"2024-04-25T13:18:32.964751+0000","last_peered":"2024-04-25T13:18:32.964751+0000","last_clean":"2024-04-25T13:18:32.964751+0000","last_became_active":"2024-04-25T13:17:40.823878+0000","last_became_peered":"2024-04-25T13:17:40.823878+0000","last_unstale":"2024-04-25T13:18:32.964751+0000","last_undegraded":"2024-04-25T13:18:32.964751+0000","last_fullsized":"2024-04-25T13:18:32.964751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:23.745510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948341+0000","last_change":"2024-04-25T13:17:39.812816+0000","last_active":"2024-04-25T13:18:32.948341+0000","last_peered":"2024-04-25T13:18:32.948341+0000","last_clean":"2024-04-25T13:18:32.948341+0000","last_became_active":"2024-04-25T13:17:39.812483+0000","last_became_peered":"2024-04-25T13:17:39.812483+0000","last_unstale":"2024-04-25T13:18:32.948341+0000","last_undegraded":"2024-04-25T13:18:32.948341+0000","last_fullsized":"2024-04-25T13:18:32.948341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:14.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189894+0000","last_change":"2024-04-25T13:17:40.824764+0000","last_active":"2024-04-25T13:18:33.189894+0000","last_peered":"2024-04-25T13:18:33.189894+0000","last_clean":"2024-04-25T13:18:33.189894+0000","last_became_active":"2024-04-25T13:17:40.824337+0000","last_became_peered":"2024-04-25T13:17:40.824337+0000","last_unstale":"2024-04-25T13:18:33.189894+0000","last_undegraded":"2024-04-25T13:18:33.189894+0000","last_fullsized":"2024-04-25T13:18:33.189894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:37.125246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.190058+0000","last_change":"2024-04-25T13:17:39.819820+0000","last_active":"2024-04-25T13:18:33.190058+0000","last_peered":"2024-04-25T13:18:33.190058+0000","last_clean":"2024-04-25T13:18:33.190058+0000","last_became_active":"2024-04-25T13:17:39.819446+0000","last_became_peered":"2024-04-25T13:17:39.819446+0000","last_unstale":"2024-04-25T13:18:33.190058+0000","last_undegraded":"2024-04-25T13:18:33.190058+0000","last_fullsized":"2024-04-25T13:18:33.190058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:36.902011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948781+0000","last_change":"2024-04-25T13:17:40.821183+0000","last_active":"2024-04-25T13:18:32.948781+0000","last_peered":"2024-04-25T13:18:32.948781+0000","last_clean":"2024-04-25T13:18:32.948781+0000","last_became_active":"2024-04-25T13:17:40.820789+0000","last_became_peered":"2024-04-25T13:17:40.820789+0000","last_unstale":"2024-04-25T13:18:32.948781+0000","last_undegraded":"2024-04-25T13:18:32.948781+0000","last_fullsized":"2024-04-25T13:18:32.948781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:56.225385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949888+0000","last_change":"2024-04-25T13:17:39.813132+0000","last_active":"2024-04-25T13:18:32.949888+0000","last_peered":"2024-04-25T13:18:32.949888+0000","last_clean":"2024-04-25T13:18:32.949888+0000","last_became_active":"2024-04-25T13:17:39.812837+0000","last_became_peered":"2024-04-25T13:17:39.812837+0000","last_unstale":"2024-04-25T13:18:32.949888+0000","last_undegraded":"2024-04-25T13:18:32.949888+0000","last_fullsized":"2024-04-25T13:18:32.949888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:55.342376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.950609+0000","last_change":"2024-04-25T13:17:40.824143+0000","last_active":"2024-04-25T13:18:32.950609+0000","last_peered":"2024-04-25T13:18:32.950609+0000","last_clean":"2024-04-25T13:18:32.950609+0000","last_became_active":"2024-04-25T13:17:40.823778+0000","last_became_peered":"2024-04-25T13:17:40.823778+0000","last_unstale":"2024-04-25T13:18:32.950609+0000","last_undegraded":"2024-04-25T13:18:32.950609+0000","last_fullsized":"2024-04-25T13:18:32.950609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:54.112957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949534+0000","last_change":"2024-04-25T13:17:39.817743+0000","last_active":"2024-04-25T13:18:32.949534+0000","last_peered":"2024-04-25T13:18:32.949534+0000","last_clean":"2024-04-25T13:18:32.949534+0000","last_became_active":"2024-04-25T13:17:39.817392+0000","last_became_peered":"2024-04-25T13:17:39.817392+0000","last_unstale":"2024-04-25T13:18:32.949534+0000","last_undegraded":"2024-04-25T13:18:32.949534+0000","last_fullsized":"2024-04-25T13:18:32.949534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.400644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.946435+0000","last_change":"2024-04-25T13:17:40.829895+0000","last_active":"2024-04-25T13:18:32.946435+0000","last_peered":"2024-04-25T13:18:32.946435+0000","last_clean":"2024-04-25T13:18:32.946435+0000","last_became_active":"2024-04-25T13:17:40.829300+0000","last_became_peered":"2024-04-25T13:17:40.829300+0000","last_unstale":"2024-04-25T13:18:32.946435+0000","last_undegraded":"2024-04-25T13:18:32.946435+0000","last_fullsized":"2024-04-25T13:18:32.946435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:49.935081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.946196+0000","last_change":"2024-04-25T13:17:39.813231+0000","last_active":"2024-04-25T13:18:32.946196+0000","last_peered":"2024-04-25T13:18:32.946196+0000","last_clean":"2024-04-25T13:18:32.946196+0000","last_became_active":"2024-04-25T13:17:39.812441+0000","last_became_peered":"2024-04-25T13:17:39.812441+0000","last_unstale":"2024-04-25T13:18:32.946196+0000","last_undegraded":"2024-04-25T13:18:32.946196+0000","last_fullsized":"2024-04-25T13:18:32.946196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:17.578986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.168914+0000","last_change":"2024-04-25T13:18:33.168914+0000","last_active":"2024-04-25T13:18:33.168914+0000","last_peered":"2024-04-25T13:18:33.168914+0000","last_clean":"2024-04-25T13:18:33.168914+0000","last_became_active":"2024-04-25T13:18:33.168620+0000","last_became_peered":"2024-04-25T13:18:33.168620+0000","last_unstale":"2024-04-25T13:18:33.168914+0000","last_undegraded":"2024-04-25T13:18:33.168914+0000","last_fullsized":"2024-04-25T13:18:33.168914+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:03.478414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332193+0000","last_change":"2024-04-25T13:17:39.809888+0000","last_active":"2024-04-25T13:18:33.332193+0000","last_peered":"2024-04-25T13:18:33.332193+0000","last_clean":"2024-04-25T13:18:33.332193+0000","last_became_active":"2024-04-25T13:17:39.809519+0000","last_became_peered":"2024-04-25T13:17:39.809519+0000","last_unstale":"2024-04-25T13:18:33.332193+0000","last_undegraded":"2024-04-25T13:18:33.332193+0000","last_fullsized":"2024-04-25T13:18:33.332193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:43.306668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.190931+0000","last_change":"2024-04-25T13:18:33.190931+0000","last_active":"2024-04-25T13:18:33.190931+0000","last_peered":"2024-04-25T13:18:33.190931+0000","last_clean":"2024-04-25T13:18:33.190931+0000","last_became_active":"2024-04-25T13:18:33.190517+0000","last_became_peered":"2024-04-25T13:18:33.190517+0000","last_unstale":"2024-04-25T13:18:33.190931+0000","last_undegraded":"2024-04-25T13:18:33.190931+0000","last_fullsized":"2024-04-25T13:18:33.190931+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:32.925276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945288+0000","last_change":"2024-04-25T13:17:39.811231+0000","last_active":"2024-04-25T13:18:32.945288+0000","last_peered":"2024-04-25T13:18:32.945288+0000","last_clean":"2024-04-25T13:18:32.945288+0000","last_became_active":"2024-04-25T13:17:39.810762+0000","last_became_peered":"2024-04-25T13:17:39.810762+0000","last_unstale":"2024-04-25T13:18:32.945288+0000","last_undegraded":"2024-04-25T13:18:32.945288+0000","last_fullsized":"2024-04-25T13:18:32.945288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:15.970042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189536+0000","last_change":"2024-04-25T13:17:40.828074+0000","last_active":"2024-04-25T13:18:33.189536+0000","last_peered":"2024-04-25T13:18:33.189536+0000","last_clean":"2024-04-25T13:18:33.189536+0000","last_became_active":"2024-04-25T13:17:40.827702+0000","last_became_peered":"2024-04-25T13:17:40.827702+0000","last_unstale":"2024-04-25T13:18:33.189536+0000","last_undegraded":"2024-04-25T13:18:33.189536+0000","last_fullsized":"2024-04-25T13:18:33.189536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:23.408908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945918+0000","last_change":"2024-04-25T13:17:39.812746+0000","last_active":"2024-04-25T13:18:32.945918+0000","last_peered":"2024-04-25T13:18:32.945918+0000","last_clean":"2024-04-25T13:18:32.945918+0000","last_became_active":"2024-04-25T13:17:39.812163+0000","last_became_peered":"2024-04-25T13:17:39.812163+0000","last_unstale":"2024-04-25T13:18:32.945918+0000","last_undegraded":"2024-04-25T13:18:32.945918+0000","last_fullsized":"2024-04-25T13:18:32.945918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:20.410328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948238+0000","last_change":"2024-04-25T13:17:40.829512+0000","last_active":"2024-04-25T13:18:32.948238+0000","last_peered":"2024-04-25T13:18:32.948238+0000","last_clean":"2024-04-25T13:18:32.948238+0000","last_became_active":"2024-04-25T13:17:40.829084+0000","last_became_peered":"2024-04-25T13:17:40.829084+0000","last_unstale":"2024-04-25T13:18:32.948238+0000","last_undegraded":"2024-04-25T13:18:32.948238+0000","last_fullsized":"2024-04-25T13:18:32.948238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:55.473018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332622+0000","last_change":"2024-04-25T13:17:39.810766+0000","last_active":"2024-04-25T13:18:33.332622+0000","last_peered":"2024-04-25T13:18:33.332622+0000","last_clean":"2024-04-25T13:18:33.332622+0000","last_became_active":"2024-04-25T13:17:39.810444+0000","last_became_peered":"2024-04-25T13:17:39.810444+0000","last_unstale":"2024-04-25T13:18:33.332622+0000","last_undegraded":"2024-04-25T13:18:33.332622+0000","last_fullsized":"2024-04-25T13:18:33.332622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:33.470819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.950846+0000","last_change":"2024-04-25T13:17:40.825574+0000","last_active":"2024-04-25T13:18:32.950846+0000","last_peered":"2024-04-25T13:18:32.950846+0000","last_clean":"2024-04-25T13:18:32.950846+0000","last_became_active":"2024-04-25T13:17:40.825220+0000","last_became_peered":"2024-04-25T13:17:40.825220+0000","last_unstale":"2024-04-25T13:18:32.950846+0000","last_undegraded":"2024-04-25T13:18:32.950846+0000","last_fullsized":"2024-04-25T13:18:32.950846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:13.941665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167496+0000","last_change":"2024-04-25T13:17:39.816040+0000","last_active":"2024-04-25T13:18:33.167496+0000","last_peered":"2024-04-25T13:18:33.167496+0000","last_clean":"2024-04-25T13:18:33.167496+0000","last_became_active":"2024-04-25T13:17:39.815746+0000","last_became_peered":"2024-04-25T13:17:39.815746+0000","last_unstale":"2024-04-25T13:18:33.167496+0000","last_undegraded":"2024-04-25T13:18:33.167496+0000","last_fullsized":"2024-04-25T13:18:33.167496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:12.068992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948704+0000","last_change":"2024-04-25T13:17:40.821926+0000","last_active":"2024-04-25T13:18:32.948704+0000","last_peered":"2024-04-25T13:18:32.948704+0000","last_clean":"2024-04-25T13:18:32.948704+0000","last_became_active":"2024-04-25T13:17:40.821280+0000","last_became_peered":"2024-04-25T13:17:40.821280+0000","last_unstale":"2024-04-25T13:18:32.948704+0000","last_undegraded":"2024-04-25T13:18:32.948704+0000","last_fullsized":"2024-04-25T13:18:32.948704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:33.662553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.946542+0000","last_change":"2024-04-25T13:17:39.819113+0000","last_active":"2024-04-25T13:18:32.946542+0000","last_peered":"2024-04-25T13:18:32.946542+0000","last_clean":"2024-04-25T13:18:32.946542+0000","last_became_active":"2024-04-25T13:17:39.818749+0000","last_became_peered":"2024-04-25T13:17:39.818749+0000","last_unstale":"2024-04-25T13:18:32.946542+0000","last_undegraded":"2024-04-25T13:18:32.946542+0000","last_fullsized":"2024-04-25T13:18:32.946542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:32.119616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.950657+0000","last_change":"2024-04-25T13:17:40.821066+0000","last_active":"2024-04-25T13:18:32.950657+0000","last_peered":"2024-04-25T13:18:32.950657+0000","last_clean":"2024-04-25T13:18:32.950657+0000","last_became_active":"2024-04-25T13:17:40.820674+0000","last_became_peered":"2024-04-25T13:17:40.820674+0000","last_unstale":"2024-04-25T13:18:32.950657+0000","last_undegraded":"2024-04-25T13:18:32.950657+0000","last_fullsized":"2024-04-25T13:18:32.950657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:01.157850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":182,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:47.939212+0000","last_change":"2024-04-25T13:17:39.811131+0000","last_active":"2024-04-25T13:18:47.939212+0000","last_peered":"2024-04-25T13:18:47.939212+0000","last_clean":"2024-04-25T13:18:47.939212+0000","last_became_active":"2024-04-25T13:17:39.810791+0000","last_became_peered":"2024-04-25T13:17:39.810791+0000","last_unstale":"2024-04-25T13:18:47.939212+0000","last_undegraded":"2024-04-25T13:18:47.939212+0000","last_fullsized":"2024-04-25T13:18:47.939212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:43.964183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":137,"num_read_kb":127,"num_write":26,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949577+0000","last_change":"2024-04-25T13:17:40.818077+0000","last_active":"2024-04-25T13:18:32.949577+0000","last_peered":"2024-04-25T13:18:32.949577+0000","last_clean":"2024-04-25T13:18:32.949577+0000","last_became_active":"2024-04-25T13:17:40.817757+0000","last_became_peered":"2024-04-25T13:17:40.817757+0000","last_unstale":"2024-04-25T13:18:32.949577+0000","last_undegraded":"2024-04-25T13:18:32.949577+0000","last_fullsized":"2024-04-25T13:18:32.949577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:17.526603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964479+0000","last_change":"2024-04-25T13:17:39.809111+0000","last_active":"2024-04-25T13:18:32.964479+0000","last_peered":"2024-04-25T13:18:32.964479+0000","last_clean":"2024-04-25T13:18:32.964479+0000","last_became_active":"2024-04-25T13:17:39.808785+0000","last_became_peered":"2024-04-25T13:17:39.808785+0000","last_unstale":"2024-04-25T13:18:32.964479+0000","last_undegraded":"2024-04-25T13:18:32.964479+0000","last_fullsized":"2024-04-25T13:18:32.964479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:41.549940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945334+0000","last_change":"2024-04-25T13:17:40.835646+0000","last_active":"2024-04-25T13:18:32.945334+0000","last_peered":"2024-04-25T13:18:32.945334+0000","last_clean":"2024-04-25T13:18:32.945334+0000","last_became_active":"2024-04-25T13:17:40.835267+0000","last_became_peered":"2024-04-25T13:17:40.835267+0000","last_unstale":"2024-04-25T13:18:32.945334+0000","last_undegraded":"2024-04-25T13:18:32.945334+0000","last_fullsized":"2024-04-25T13:18:32.945334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:05.811651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.946146+0000","last_change":"2024-04-25T13:17:39.813006+0000","last_active":"2024-04-25T13:18:32.946146+0000","last_peered":"2024-04-25T13:18:32.946146+0000","last_clean":"2024-04-25T13:18:32.946146+0000","last_became_active":"2024-04-25T13:17:39.812635+0000","last_became_peered":"2024-04-25T13:17:39.812635+0000","last_unstale":"2024-04-25T13:18:32.946146+0000","last_undegraded":"2024-04-25T13:18:32.946146+0000","last_fullsized":"2024-04-25T13:18:32.946146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.922568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167291+0000","last_change":"2024-04-25T13:17:40.833365+0000","last_active":"2024-04-25T13:18:33.167291+0000","last_peered":"2024-04-25T13:18:33.167291+0000","last_clean":"2024-04-25T13:18:33.167291+0000","last_became_active":"2024-04-25T13:17:40.832927+0000","last_became_peered":"2024-04-25T13:17:40.832927+0000","last_unstale":"2024-04-25T13:18:33.167291+0000","last_undegraded":"2024-04-25T13:18:33.167291+0000","last_fullsized":"2024-04-25T13:18:33.167291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.622153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.289871+0000","last_change":"2024-04-25T13:17:39.810983+0000","last_active":"2024-04-25T13:18:33.289871+0000","last_peered":"2024-04-25T13:18:33.289871+0000","last_clean":"2024-04-25T13:18:33.289871+0000","last_became_active":"2024-04-25T13:17:39.810366+0000","last_became_peered":"2024-04-25T13:17:39.810366+0000","last_unstale":"2024-04-25T13:18:33.289871+0000","last_undegraded":"2024-04-25T13:18:33.289871+0000","last_fullsized":"2024-04-25T13:18:33.289871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:33.636924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.166706+0000","last_change":"2024-04-25T13:17:40.818301+0000","last_active":"2024-04-25T13:18:33.166706+0000","last_peered":"2024-04-25T13:18:33.166706+0000","last_clean":"2024-04-25T13:18:33.166706+0000","last_became_active":"2024-04-25T13:17:40.817903+0000","last_became_peered":"2024-04-25T13:17:40.817903+0000","last_unstale":"2024-04-25T13:18:33.166706+0000","last_undegraded":"2024-04-25T13:18:33.166706+0000","last_fullsized":"2024-04-25T13:18:33.166706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:40.913771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948966+0000","last_change":"2024-04-25T13:17:39.820411+0000","last_active":"2024-04-25T13:18:32.948966+0000","last_peered":"2024-04-25T13:18:32.948966+0000","last_clean":"2024-04-25T13:18:32.948966+0000","last_became_active":"2024-04-25T13:17:39.820051+0000","last_became_peered":"2024-04-25T13:17:39.820051+0000","last_unstale":"2024-04-25T13:18:32.948966+0000","last_undegraded":"2024-04-25T13:18:32.948966+0000","last_fullsized":"2024-04-25T13:18:32.948966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:08.931024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.291455+0000","last_change":"2024-04-25T13:18:33.291044+0000","last_active":"2024-04-25T13:18:33.291455+0000","last_peered":"2024-04-25T13:18:33.291455+0000","last_clean":"2024-04-25T13:18:33.291455+0000","last_became_active":"2024-04-25T13:18:33.290673+0000","last_became_peered":"2024-04-25T13:18:33.290673+0000","last_unstale":"2024-04-25T13:18:33.291455+0000","last_undegraded":"2024-04-25T13:18:33.291455+0000","last_fullsized":"2024-04-25T13:18:33.291455+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:41.657993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949901+0000","last_change":"2024-04-25T13:17:39.812160+0000","last_active":"2024-04-25T13:18:32.949901+0000","last_peered":"2024-04-25T13:18:32.949901+0000","last_clean":"2024-04-25T13:18:32.949901+0000","last_became_active":"2024-04-25T13:17:39.811846+0000","last_became_peered":"2024-04-25T13:17:39.811846+0000","last_unstale":"2024-04-25T13:18:32.949901+0000","last_undegraded":"2024-04-25T13:18:32.949901+0000","last_fullsized":"2024-04-25T13:18:32.949901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:20.146245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.951554+0000","last_change":"2024-04-25T13:18:32.951554+0000","last_active":"2024-04-25T13:18:32.951554+0000","last_peered":"2024-04-25T13:18:32.951554+0000","last_clean":"2024-04-25T13:18:32.951554+0000","last_became_active":"2024-04-25T13:18:32.951082+0000","last_became_peered":"2024-04-25T13:18:32.951082+0000","last_unstale":"2024-04-25T13:18:32.951554+0000","last_undegraded":"2024-04-25T13:18:32.951554+0000","last_fullsized":"2024-04-25T13:18:32.951554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:41.550497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948192+0000","last_change":"2024-04-25T13:17:39.819540+0000","last_active":"2024-04-25T13:18:32.948192+0000","last_peered":"2024-04-25T13:18:32.948192+0000","last_clean":"2024-04-25T13:18:32.948192+0000","last_became_active":"2024-04-25T13:17:39.818942+0000","last_became_peered":"2024-04-25T13:17:39.818942+0000","last_unstale":"2024-04-25T13:18:32.948192+0000","last_undegraded":"2024-04-25T13:18:32.948192+0000","last_fullsized":"2024-04-25T13:18:32.948192+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:51.300087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332613+0000","last_change":"2024-04-25T13:17:40.818872+0000","last_active":"2024-04-25T13:18:33.332613+0000","last_peered":"2024-04-25T13:18:33.332613+0000","last_clean":"2024-04-25T13:18:33.332613+0000","last_became_active":"2024-04-25T13:17:40.818522+0000","last_became_peered":"2024-04-25T13:17:40.818522+0000","last_unstale":"2024-04-25T13:18:33.332613+0000","last_undegraded":"2024-04-25T13:18:33.332613+0000","last_fullsized":"2024-04-25T13:18:33.332613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.928782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167703+0000","last_change":"2024-04-25T13:17:39.815555+0000","last_active":"2024-04-25T13:18:33.167703+0000","last_peered":"2024-04-25T13:18:33.167703+0000","last_clean":"2024-04-25T13:18:33.167703+0000","last_became_active":"2024-04-25T13:17:39.815249+0000","last_became_peered":"2024-04-25T13:17:39.815249+0000","last_unstale":"2024-04-25T13:18:33.167703+0000","last_undegraded":"2024-04-25T13:18:33.167703+0000","last_fullsized":"2024-04-25T13:18:33.167703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:40.723562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332505+0000","last_change":"2024-04-25T13:17:40.821643+0000","last_active":"2024-04-25T13:18:33.332505+0000","last_peered":"2024-04-25T13:18:33.332505+0000","last_clean":"2024-04-25T13:18:33.332505+0000","last_became_active":"2024-04-25T13:17:40.820715+0000","last_became_peered":"2024-04-25T13:17:40.820715+0000","last_unstale":"2024-04-25T13:18:33.332505+0000","last_undegraded":"2024-04-25T13:18:33.332505+0000","last_fullsized":"2024-04-25T13:18:33.332505+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:54.046030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.965993+0000","last_change":"2024-04-25T13:18:32.965993+0000","last_active":"2024-04-25T13:18:32.965993+0000","last_peered":"2024-04-25T13:18:32.965993+0000","last_clean":"2024-04-25T13:18:32.965993+0000","last_became_active":"2024-04-25T13:18:32.965546+0000","last_became_peered":"2024-04-25T13:18:32.965546+0000","last_unstale":"2024-04-25T13:18:32.965993+0000","last_undegraded":"2024-04-25T13:18:32.965993+0000","last_fullsized":"2024-04-25T13:18:32.965993+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:10.496431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945162+0000","last_change":"2024-04-25T13:17:39.820658+0000","last_active":"2024-04-25T13:18:32.945162+0000","last_peered":"2024-04-25T13:18:32.945162+0000","last_clean":"2024-04-25T13:18:32.945162+0000","last_became_active":"2024-04-25T13:17:39.820122+0000","last_became_peered":"2024-04-25T13:17:39.820122+0000","last_unstale":"2024-04-25T13:18:32.945162+0000","last_undegraded":"2024-04-25T13:18:32.945162+0000","last_fullsized":"2024-04-25T13:18:32.945162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:04.655982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189662+0000","last_change":"2024-04-25T13:17:40.827220+0000","last_active":"2024-04-25T13:18:33.189662+0000","last_peered":"2024-04-25T13:18:33.189662+0000","last_clean":"2024-04-25T13:18:33.189662+0000","last_became_active":"2024-04-25T13:17:40.826517+0000","last_became_peered":"2024-04-25T13:17:40.826517+0000","last_unstale":"2024-04-25T13:18:33.189662+0000","last_undegraded":"2024-04-25T13:18:33.189662+0000","last_fullsized":"2024-04-25T13:18:33.189662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:30.405644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.289527+0000","last_change":"2024-04-25T13:17:39.813893+0000","last_active":"2024-04-25T13:18:33.289527+0000","last_peered":"2024-04-25T13:18:33.289527+0000","last_clean":"2024-04-25T13:18:33.289527+0000","last_became_active":"2024-04-25T13:17:39.812939+0000","last_became_peered":"2024-04-25T13:17:39.812939+0000","last_unstale":"2024-04-25T13:18:33.289527+0000","last_undegraded":"2024-04-25T13:18:33.289527+0000","last_fullsized":"2024-04-25T13:18:33.289527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.033913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.166824+0000","last_change":"2024-04-25T13:16:30.734601+0000","last_active":"2024-04-25T13:18:33.166824+0000","last_peered":"2024-04-25T13:18:33.166824+0000","last_clean":"2024-04-25T13:18:33.166824+0000","last_became_active":"2024-04-25T13:16:30.733987+0000","last_became_peered":"2024-04-25T13:16:30.733987+0000","last_unstale":"2024-04-25T13:18:33.166824+0000","last_undegraded":"2024-04-25T13:18:33.166824+0000","last_fullsized":"2024-04-25T13:18:33.166824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:13:02.186391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:13:02.186391+0000","last_clean_scrub_stamp":"2024-04-25T13:13:02.186391+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:41.669205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.965694+0000","last_change":"2024-04-25T13:18:32.965694+0000","last_active":"2024-04-25T13:18:32.965694+0000","last_peered":"2024-04-25T13:18:32.965694+0000","last_clean":"2024-04-25T13:18:32.965694+0000","last_became_active":"2024-04-25T13:18:32.965279+0000","last_became_peered":"2024-04-25T13:18:32.965279+0000","last_unstale":"2024-04-25T13:18:32.965694+0000","last_undegraded":"2024-04-25T13:18:32.965694+0000","last_fullsized":"2024-04-25T13:18:32.965694+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:28.962446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.166943+0000","last_change":"2024-04-25T13:17:39.809530+0000","last_active":"2024-04-25T13:18:33.166943+0000","last_peered":"2024-04-25T13:18:33.166943+0000","last_clean":"2024-04-25T13:18:33.166943+0000","last_became_active":"2024-04-25T13:17:39.809198+0000","last_became_peered":"2024-04-25T13:17:39.809198+0000","last_unstale":"2024-04-25T13:18:33.166943+0000","last_undegraded":"2024-04-25T13:18:33.166943+0000","last_fullsized":"2024-04-25T13:18:33.166943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:34.397912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949700+0000","last_change":"2024-04-25T13:17:39.819607+0000","last_active":"2024-04-25T13:18:32.949700+0000","last_peered":"2024-04-25T13:18:32.949700+0000","last_clean":"2024-04-25T13:18:32.949700+0000","last_became_active":"2024-04-25T13:17:39.819273+0000","last_became_peered":"2024-04-25T13:17:39.819273+0000","last_unstale":"2024-04-25T13:18:32.949700+0000","last_undegraded":"2024-04-25T13:18:32.949700+0000","last_fullsized":"2024-04-25T13:18:32.949700+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:39.414085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945248+0000","last_change":"2024-04-25T13:17:40.824554+0000","last_active":"2024-04-25T13:18:32.945248+0000","last_peered":"2024-04-25T13:18:32.945248+0000","last_clean":"2024-04-25T13:18:32.945248+0000","last_became_active":"2024-04-25T13:17:40.824249+0000","last_became_peered":"2024-04-25T13:17:40.824249+0000","last_unstale":"2024-04-25T13:18:32.945248+0000","last_undegraded":"2024-04-25T13:18:32.945248+0000","last_fullsized":"2024-04-25T13:18:32.945248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:17.282312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948329+0000","last_change":"2024-04-25T13:17:39.820095+0000","last_active":"2024-04-25T13:18:32.948329+0000","last_peered":"2024-04-25T13:18:32.948329+0000","last_clean":"2024-04-25T13:18:32.948329+0000","last_became_active":"2024-04-25T13:17:39.819774+0000","last_became_peered":"2024-04-25T13:17:39.819774+0000","last_unstale":"2024-04-25T13:18:32.948329+0000","last_undegraded":"2024-04-25T13:18:32.948329+0000","last_fullsized":"2024-04-25T13:18:32.948329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:09.154958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189963+0000","last_change":"2024-04-25T13:17:40.830202+0000","last_active":"2024-04-25T13:18:33.189963+0000","last_peered":"2024-04-25T13:18:33.189963+0000","last_clean":"2024-04-25T13:18:33.189963+0000","last_became_active":"2024-04-25T13:17:40.829814+0000","last_became_peered":"2024-04-25T13:17:40.829814+0000","last_unstale":"2024-04-25T13:18:33.189963+0000","last_undegraded":"2024-04-25T13:18:33.189963+0000","last_fullsized":"2024-04-25T13:18:33.189963+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:05.492378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.289947+0000","last_change":"2024-04-25T13:17:39.814680+0000","last_active":"2024-04-25T13:18:33.289947+0000","last_peered":"2024-04-25T13:18:33.289947+0000","last_clean":"2024-04-25T13:18:33.289947+0000","last_became_active":"2024-04-25T13:17:39.814324+0000","last_became_peered":"2024-04-25T13:17:39.814324+0000","last_unstale":"2024-04-25T13:18:33.289947+0000","last_undegraded":"2024-04-25T13:18:33.289947+0000","last_fullsized":"2024-04-25T13:18:33.289947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:42.818060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.290964+0000","last_change":"2024-04-25T13:18:33.290964+0000","last_active":"2024-04-25T13:18:33.290964+0000","last_peered":"2024-04-25T13:18:33.290964+0000","last_clean":"2024-04-25T13:18:33.290964+0000","last_became_active":"2024-04-25T13:18:33.290420+0000","last_became_peered":"2024-04-25T13:18:33.290420+0000","last_unstale":"2024-04-25T13:18:33.290964+0000","last_undegraded":"2024-04-25T13:18:33.290964+0000","last_fullsized":"2024-04-25T13:18:33.290964+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:34.696247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.950358+0000","last_change":"2024-04-25T13:17:39.820908+0000","last_active":"2024-04-25T13:18:32.950358+0000","last_peered":"2024-04-25T13:18:32.950358+0000","last_clean":"2024-04-25T13:18:32.950358+0000","last_became_active":"2024-04-25T13:17:39.820612+0000","last_became_peered":"2024-04-25T13:17:39.820612+0000","last_unstale":"2024-04-25T13:18:32.950358+0000","last_undegraded":"2024-04-25T13:18:32.950358+0000","last_fullsized":"2024-04-25T13:18:32.950358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:15.170079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948951+0000","last_change":"2024-04-25T13:17:40.817232+0000","last_active":"2024-04-25T13:18:32.948951+0000","last_peered":"2024-04-25T13:18:32.948951+0000","last_clean":"2024-04-25T13:18:32.948951+0000","last_became_active":"2024-04-25T13:17:40.816895+0000","last_became_peered":"2024-04-25T13:17:40.816895+0000","last_unstale":"2024-04-25T13:18:32.948951+0000","last_undegraded":"2024-04-25T13:18:32.948951+0000","last_fullsized":"2024-04-25T13:18:32.948951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:28.772948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189546+0000","last_change":"2024-04-25T13:17:39.816586+0000","last_active":"2024-04-25T13:18:33.189546+0000","last_peered":"2024-04-25T13:18:33.189546+0000","last_clean":"2024-04-25T13:18:33.189546+0000","last_became_active":"2024-04-25T13:17:39.815712+0000","last_became_peered":"2024-04-25T13:17:39.815712+0000","last_unstale":"2024-04-25T13:18:33.189546+0000","last_undegraded":"2024-04-25T13:18:33.189546+0000","last_fullsized":"2024-04-25T13:18:33.189546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.772789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948386+0000","last_change":"2024-04-25T13:17:40.824505+0000","last_active":"2024-04-25T13:18:32.948386+0000","last_peered":"2024-04-25T13:18:32.948386+0000","last_clean":"2024-04-25T13:18:32.948386+0000","last_became_active":"2024-04-25T13:17:40.824197+0000","last_became_peered":"2024-04-25T13:17:40.824197+0000","last_unstale":"2024-04-25T13:18:32.948386+0000","last_undegraded":"2024-04-25T13:18:32.948386+0000","last_fullsized":"2024-04-25T13:18:32.948386+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:17.484269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964352+0000","last_change":"2024-04-25T13:17:39.810242+0000","last_active":"2024-04-25T13:18:32.964352+0000","last_peered":"2024-04-25T13:18:32.964352+0000","last_clean":"2024-04-25T13:18:32.964352+0000","last_became_active":"2024-04-25T13:17:39.809547+0000","last_became_peered":"2024-04-25T13:17:39.809547+0000","last_unstale":"2024-04-25T13:18:32.964352+0000","last_undegraded":"2024-04-25T13:18:32.964352+0000","last_fullsized":"2024-04-25T13:18:32.964352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.615321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189756+0000","last_change":"2024-04-25T13:17:40.822114+0000","last_active":"2024-04-25T13:18:33.189756+0000","last_peered":"2024-04-25T13:18:33.189756+0000","last_clean":"2024-04-25T13:18:33.189756+0000","last_became_active":"2024-04-25T13:17:40.821517+0000","last_became_peered":"2024-04-25T13:17:40.821517+0000","last_unstale":"2024-04-25T13:18:33.189756+0000","last_undegraded":"2024-04-25T13:18:33.189756+0000","last_fullsized":"2024-04-25T13:18:33.189756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:07.225119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949571+0000","last_change":"2024-04-25T13:17:39.812135+0000","last_active":"2024-04-25T13:18:32.949571+0000","last_peered":"2024-04-25T13:18:32.949571+0000","last_clean":"2024-04-25T13:18:32.949571+0000","last_became_active":"2024-04-25T13:17:39.811836+0000","last_became_peered":"2024-04-25T13:17:39.811836+0000","last_unstale":"2024-04-25T13:18:32.949571+0000","last_undegraded":"2024-04-25T13:18:32.949571+0000","last_fullsized":"2024-04-25T13:18:32.949571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:00.692725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167708+0000","last_change":"2024-04-25T13:17:40.830160+0000","last_active":"2024-04-25T13:18:33.167708+0000","last_peered":"2024-04-25T13:18:33.167708+0000","last_clean":"2024-04-25T13:18:33.167708+0000","last_became_active":"2024-04-25T13:17:40.829773+0000","last_became_peered":"2024-04-25T13:17:40.829773+0000","last_unstale":"2024-04-25T13:18:33.167708+0000","last_undegraded":"2024-04-25T13:18:33.167708+0000","last_fullsized":"2024-04-25T13:18:33.167708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:24.515333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964731+0000","last_change":"2024-04-25T13:17:39.819011+0000","last_active":"2024-04-25T13:18:32.964731+0000","last_peered":"2024-04-25T13:18:32.964731+0000","last_clean":"2024-04-25T13:18:32.964731+0000","last_became_active":"2024-04-25T13:17:39.818656+0000","last_became_peered":"2024-04-25T13:17:39.818656+0000","last_unstale":"2024-04-25T13:18:32.964731+0000","last_undegraded":"2024-04-25T13:18:32.964731+0000","last_fullsized":"2024-04-25T13:18:32.964731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:20.507500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167790+0000","last_change":"2024-04-25T13:17:40.829595+0000","last_active":"2024-04-25T13:18:33.167790+0000","last_peered":"2024-04-25T13:18:33.167790+0000","last_clean":"2024-04-25T13:18:33.167790+0000","last_became_active":"2024-04-25T13:17:40.827707+0000","last_became_peered":"2024-04-25T13:17:40.827707+0000","last_unstale":"2024-04-25T13:18:33.167790+0000","last_undegraded":"2024-04-25T13:18:33.167790+0000","last_fullsized":"2024-04-25T13:18:33.167790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:49.002629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.946543+0000","last_change":"2024-04-25T13:17:39.809740+0000","last_active":"2024-04-25T13:18:32.946543+0000","last_peered":"2024-04-25T13:18:32.946543+0000","last_clean":"2024-04-25T13:18:32.946543+0000","last_became_active":"2024-04-25T13:17:39.809407+0000","last_became_peered":"2024-04-25T13:17:39.809407+0000","last_unstale":"2024-04-25T13:18:32.946543+0000","last_undegraded":"2024-04-25T13:18:32.946543+0000","last_fullsized":"2024-04-25T13:18:32.946543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:16.732755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167519+0000","last_change":"2024-04-25T13:17:40.825573+0000","last_active":"2024-04-25T13:18:33.167519+0000","last_peered":"2024-04-25T13:18:33.167519+0000","last_clean":"2024-04-25T13:18:33.167519+0000","last_became_active":"2024-04-25T13:17:40.825181+0000","last_became_peered":"2024-04-25T13:17:40.825181+0000","last_unstale":"2024-04-25T13:18:33.167519+0000","last_undegraded":"2024-04-25T13:18:33.167519+0000","last_fullsized":"2024-04-25T13:18:33.167519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:06.446494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945434+0000","last_change":"2024-04-25T13:17:39.812458+0000","last_active":"2024-04-25T13:18:32.945434+0000","last_peered":"2024-04-25T13:18:32.945434+0000","last_clean":"2024-04-25T13:18:32.945434+0000","last_became_active":"2024-04-25T13:17:39.812092+0000","last_became_peered":"2024-04-25T13:17:39.812092+0000","last_unstale":"2024-04-25T13:18:32.945434+0000","last_undegraded":"2024-04-25T13:18:32.945434+0000","last_fullsized":"2024-04-25T13:18:32.945434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:07.210444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167636+0000","last_change":"2024-04-25T13:17:40.821510+0000","last_active":"2024-04-25T13:18:33.167636+0000","last_peered":"2024-04-25T13:18:33.167636+0000","last_clean":"2024-04-25T13:18:33.167636+0000","last_became_active":"2024-04-25T13:17:40.818897+0000","last_became_peered":"2024-04-25T13:17:40.818897+0000","last_unstale":"2024-04-25T13:18:33.167636+0000","last_undegraded":"2024-04-25T13:18:33.167636+0000","last_fullsized":"2024-04-25T13:18:33.167636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:15.215991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948406+0000","last_change":"2024-04-25T13:17:40.823789+0000","last_active":"2024-04-25T13:18:32.948406+0000","last_peered":"2024-04-25T13:18:32.948406+0000","last_clean":"2024-04-25T13:18:32.948406+0000","last_became_active":"2024-04-25T13:17:40.823424+0000","last_became_peered":"2024-04-25T13:17:40.823424+0000","last_unstale":"2024-04-25T13:18:32.948406+0000","last_undegraded":"2024-04-25T13:18:32.948406+0000","last_fullsized":"2024-04-25T13:18:32.948406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:50.022286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964529+0000","last_change":"2024-04-25T13:17:40.827085+0000","last_active":"2024-04-25T13:18:32.964529+0000","last_peered":"2024-04-25T13:18:32.964529+0000","last_clean":"2024-04-25T13:18:32.964529+0000","last_became_active":"2024-04-25T13:17:40.826760+0000","last_became_peered":"2024-04-25T13:17:40.826760+0000","last_unstale":"2024-04-25T13:18:32.964529+0000","last_undegraded":"2024-04-25T13:18:32.964529+0000","last_fullsized":"2024-04-25T13:18:32.964529+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:46.662788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964350+0000","last_change":"2024-04-25T13:17:40.824283+0000","last_active":"2024-04-25T13:18:32.964350+0000","last_peered":"2024-04-25T13:18:32.964350+0000","last_clean":"2024-04-25T13:18:32.964350+0000","last_became_active":"2024-04-25T13:17:40.823914+0000","last_became_peered":"2024-04-25T13:17:40.823914+0000","last_unstale":"2024-04-25T13:18:32.964350+0000","last_undegraded":"2024-04-25T13:18:32.964350+0000","last_fullsized":"2024-04-25T13:18:32.964350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:01.158295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949041+0000","last_change":"2024-04-25T13:17:40.822003+0000","last_active":"2024-04-25T13:18:32.949041+0000","last_peered":"2024-04-25T13:18:32.949041+0000","last_clean":"2024-04-25T13:18:32.949041+0000","last_became_active":"2024-04-25T13:17:40.821453+0000","last_became_peered":"2024-04-25T13:17:40.821453+0000","last_unstale":"2024-04-25T13:18:32.949041+0000","last_undegraded":"2024-04-25T13:18:32.949041+0000","last_fullsized":"2024-04-25T13:18:32.949041+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:34.474332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945558+0000","last_change":"2024-04-25T13:17:40.832449+0000","last_active":"2024-04-25T13:18:32.945558+0000","last_peered":"2024-04-25T13:18:32.945558+0000","last_clean":"2024-04-25T13:18:32.945558+0000","last_became_active":"2024-04-25T13:17:40.832079+0000","last_became_peered":"2024-04-25T13:17:40.832079+0000","last_unstale":"2024-04-25T13:18:32.945558+0000","last_undegraded":"2024-04-25T13:18:32.945558+0000","last_fullsized":"2024-04-25T13:18:32.945558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:44.537129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.951355+0000","last_change":"2024-04-25T13:18:32.951355+0000","last_active":"2024-04-25T13:18:32.951355+0000","last_peered":"2024-04-25T13:18:32.951355+0000","last_clean":"2024-04-25T13:18:32.951355+0000","last_became_active":"2024-04-25T13:18:32.950863+0000","last_became_peered":"2024-04-25T13:18:32.950863+0000","last_unstale":"2024-04-25T13:18:32.951355+0000","last_undegraded":"2024-04-25T13:18:32.951355+0000","last_fullsized":"2024-04-25T13:18:32.951355+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:44.837228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964929+0000","last_change":"2024-04-25T13:17:40.821024+0000","last_active":"2024-04-25T13:18:32.964929+0000","last_peered":"2024-04-25T13:18:32.964929+0000","last_clean":"2024-04-25T13:18:32.964929+0000","last_became_active":"2024-04-25T13:17:40.820671+0000","last_became_peered":"2024-04-25T13:17:40.820671+0000","last_unstale":"2024-04-25T13:18:32.964929+0000","last_undegraded":"2024-04-25T13:18:32.964929+0000","last_fullsized":"2024-04-25T13:18:32.964929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:57.315116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189806+0000","last_change":"2024-04-25T13:17:40.837131+0000","last_active":"2024-04-25T13:18:33.189806+0000","last_peered":"2024-04-25T13:18:33.189806+0000","last_clean":"2024-04-25T13:18:33.189806+0000","last_became_active":"2024-04-25T13:17:40.836784+0000","last_became_peered":"2024-04-25T13:17:40.836784+0000","last_unstale":"2024-04-25T13:18:33.189806+0000","last_undegraded":"2024-04-25T13:18:33.189806+0000","last_fullsized":"2024-04-25T13:18:33.189806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:41.818001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945994+0000","last_change":"2024-04-25T13:17:40.827149+0000","last_active":"2024-04-25T13:18:32.945994+0000","last_peered":"2024-04-25T13:18:32.945994+0000","last_clean":"2024-04-25T13:18:32.945994+0000","last_became_active":"2024-04-25T13:17:40.826800+0000","last_became_peered":"2024-04-25T13:17:40.826800+0000","last_unstale":"2024-04-25T13:18:32.945994+0000","last_undegraded":"2024-04-25T13:18:32.945994+0000","last_fullsized":"2024-04-25T13:18:32.945994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:46.211785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189470+0000","last_change":"2024-04-25T13:17:40.825028+0000","last_active":"2024-04-25T13:18:33.189470+0000","last_peered":"2024-04-25T13:18:33.189470+0000","last_clean":"2024-04-25T13:18:33.189470+0000","last_became_active":"2024-04-25T13:17:40.824614+0000","last_became_peered":"2024-04-25T13:17:40.824614+0000","last_unstale":"2024-04-25T13:18:33.189470+0000","last_undegraded":"2024-04-25T13:18:33.189470+0000","last_fullsized":"2024-04-25T13:18:33.189470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:07.629080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.169245+0000","last_change":"2024-04-25T13:18:33.169245+0000","last_active":"2024-04-25T13:18:33.169245+0000","last_peered":"2024-04-25T13:18:33.169245+0000","last_clean":"2024-04-25T13:18:33.169245+0000","last_became_active":"2024-04-25T13:18:33.168617+0000","last_became_peered":"2024-04-25T13:18:33.168617+0000","last_unstale":"2024-04-25T13:18:33.169245+0000","last_undegraded":"2024-04-25T13:18:33.169245+0000","last_fullsized":"2024-04-25T13:18:33.169245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:56.190064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945433+0000","last_change":"2024-04-25T13:17:40.835190+0000","last_active":"2024-04-25T13:18:32.945433+0000","last_peered":"2024-04-25T13:18:32.945433+0000","last_clean":"2024-04-25T13:18:32.945433+0000","last_became_active":"2024-04-25T13:17:40.834847+0000","last_became_peered":"2024-04-25T13:17:40.834847+0000","last_unstale":"2024-04-25T13:18:32.945433+0000","last_undegraded":"2024-04-25T13:18:32.945433+0000","last_fullsized":"2024-04-25T13:18:32.945433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:18.217464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948768+0000","last_change":"2024-04-25T13:17:40.827578+0000","last_active":"2024-04-25T13:18:32.948768+0000","last_peered":"2024-04-25T13:18:32.948768+0000","last_clean":"2024-04-25T13:18:32.948768+0000","last_became_active":"2024-04-25T13:17:40.827256+0000","last_became_peered":"2024-04-25T13:17:40.827256+0000","last_unstale":"2024-04-25T13:18:32.948768+0000","last_undegraded":"2024-04-25T13:18:32.948768+0000","last_fullsized":"2024-04-25T13:18:32.948768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:40.738629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61844,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":143,"num_read_kb":133,"num_write":75,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":185532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285595,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":469141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547231,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1120866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":469723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005099,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":469745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266735,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1060208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430262,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":469258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":530187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":469231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":530221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117061,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":469479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855424,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":469163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691898,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1585152,"data_stored":1059808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:18:54.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:18:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:54 smithi066 ceph-mon[33137]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:18:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:54 smithi066 ceph-mon[33137]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:54 smithi066 ceph-mon[33137]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:54.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:54.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:54.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:54 smithi136 ceph-mon[33038]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:18:54.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:54 smithi136 ceph-mon[33038]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:54.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:54.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:54.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:54 smithi136 ceph-mon[33038]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:54.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:54.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:54.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:54 smithi031 ceph-mon[29457]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:18:54.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:54 smithi031 ceph-mon[29457]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:18:54.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:54.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:54.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:54 smithi031 ceph-mon[29457]: from='mgr.14148 172.21.15.31:0/1284902528' entity='mgr.x' 2024-04-25T13:18:56.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.183+0000 7f1ddf021700 1 -- 172.21.15.31:0/3491336346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dd8103480 msgr2=0x7f1dd810f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:56.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.183+0000 7f1ddf021700 1 --2- 172.21.15.31:0/3491336346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dd8103480 0x7f1dd810f8d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000b0a0 tx=0x7f1dd000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.183+0000 7f1ddf021700 1 -- 172.21.15.31:0/3491336346 shutdown_connections 2024-04-25T13:18:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.183+0000 7f1ddf021700 1 --2- 172.21.15.31:0/3491336346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dd8103480 0x7f1dd810f8d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.183+0000 7f1ddf021700 1 --2- 172.21.15.31:0/3491336346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd8108a70 0x7f1dd8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.183+0000 7f1ddf021700 1 --2- 172.21.15.31:0/3491336346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dd8102ac0 0x7f1dd8102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.183+0000 7f1ddf021700 1 -- 172.21.15.31:0/3491336346 >> 172.21.15.31:0/3491336346 conn(0x7f1dd8079bb0 msgr2=0x7f1dd8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.183+0000 7f1ddf021700 1 -- 172.21.15.31:0/3491336346 shutdown_connections 2024-04-25T13:18:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.183+0000 7f1ddf021700 1 -- 172.21.15.31:0/3491336346 wait complete. 2024-04-25T13:18:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.183+0000 7f1ddf021700 1 Processor -- start 2024-04-25T13:18:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddf021700 1 -- start start 2024-04-25T13:18:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddf021700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dd8102ac0 0x7f1dd8199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddf021700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd8103480 0x7f1dd8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddf021700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dd8108a70 0x7f1dd819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddf021700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8112c00 con 0x7f1dd8102ac0 2024-04-25T13:18:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddf021700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8112a80 con 0x7f1dd8103480 2024-04-25T13:18:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddf021700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8112d80 con 0x7f1dd8108a70 2024-04-25T13:18:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddcdbd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dd8102ac0 0x7f1dd8199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddcdbd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dd8102ac0 0x7f1dd8199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58152/0 (socket says 172.21.15.31:58152) 2024-04-25T13:18:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddcdbd700 1 -- 172.21.15.31:0/641886012 learned_addr learned my addr 172.21.15.31:0/641886012 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddd5be700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dd8108a70 0x7f1dd819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1dd7fff700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd8103480 0x7f1dd8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddcdbd700 1 -- 172.21.15.31:0/641886012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dd8108a70 msgr2=0x7f1dd819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddcdbd700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dd8108a70 0x7f1dd819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddcdbd700 1 -- 172.21.15.31:0/641886012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd8103480 msgr2=0x7f1dd8199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddcdbd700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd8103480 0x7f1dd8199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.184+0000 7f1ddcdbd700 1 -- 172.21.15.31:0/641886012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd819e140 con 0x7f1dd8102ac0 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.185+0000 7f1ddcdbd700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dd8102ac0 0x7f1dd8199130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d950 tx=0x7f1dcc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.185+0000 7f1dd5ffb700 1 -- 172.21.15.31:0/641886012 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc016070 con 0x7f1dd8102ac0 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.185+0000 7f1ddf021700 1 -- 172.21.15.31:0/641886012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd819e430 con 0x7f1dd8102ac0 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.185+0000 7f1dd5ffb700 1 -- 172.21.15.31:0/641886012 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dcc00fd40 con 0x7f1dd8102ac0 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.185+0000 7f1dd5ffb700 1 -- 172.21.15.31:0/641886012 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00e3b0 con 0x7f1dd8102ac0 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.185+0000 7f1ddf021700 1 -- 172.21.15.31:0/641886012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd81a5cb0 con 0x7f1dd8102ac0 2024-04-25T13:18:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.186+0000 7f1ddf021700 1 -- 172.21.15.31:0/641886012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd8069730 con 0x7f1dd8102ac0 2024-04-25T13:18:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.187+0000 7f1dd5ffb700 1 -- 172.21.15.31:0/641886012 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1dcc019070 con 0x7f1dd8102ac0 2024-04-25T13:18:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.187+0000 7f1dd5ffb700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1dc80a3cd0 0x7f1dc80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.188+0000 7f1dd7fff700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1dc80a3cd0 0x7f1dc80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.188+0000 7f1dd5ffb700 1 -- 172.21.15.31:0/641886012 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f1dcc0cffe0 con 0x7f1dd8102ac0 2024-04-25T13:18:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.188+0000 7f1dd7fff700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1dc80a3cd0 0x7f1dc80a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1dd807a2b0 tx=0x7f1dc400a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:56.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.192+0000 7f1dd5ffb700 1 -- 172.21.15.31:0/641886012 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1dcc09c840 con 0x7f1dd8102ac0 2024-04-25T13:18:56.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.398+0000 7f1ddf021700 1 -- 172.21.15.31:0/641886012 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1dd8062e80 con 0x7f1dc80a3cd0 2024-04-25T13:18:56.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.405+0000 7f1dd5ffb700 1 -- 172.21.15.31:0/641886012 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258632 (secure 0 0 0) 0x7f1dd8062e80 con 0x7f1dc80a3cd0 2024-04-25T13:18:56.409 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:18:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 -- 172.21.15.31:0/641886012 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1dc80a3cd0 msgr2=0x7f1dc80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1dc80a3cd0 0x7f1dc80a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1dd807a2b0 tx=0x7f1dc400a300 comp rx=0 tx=0).stop 2024-04-25T13:18:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 -- 172.21.15.31:0/641886012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dd8102ac0 msgr2=0x7f1dd8199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dd8102ac0 0x7f1dd8199130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d950 tx=0x7f1dcc00dc60 comp rx=0 tx=0).stop 2024-04-25T13:18:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 -- 172.21.15.31:0/641886012 shutdown_connections 2024-04-25T13:18:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1dc80a3cd0 0x7f1dc80a6190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:56.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dd8108a70 0x7f1dd819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dd8102ac0 0x7f1dd8199130 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 --2- 172.21.15.31:0/641886012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd8103480 0x7f1dd8199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 -- 172.21.15.31:0/641886012 >> 172.21.15.31:0/641886012 conn(0x7f1dd8079bb0 msgr2=0x7f1dd8101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 -- 172.21.15.31:0/641886012 shutdown_connections 2024-04-25T13:18:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:56.408+0000 7f1ddf021700 1 -- 172.21.15.31:0/641886012 wait complete. 2024-04-25T13:18:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-25T13:18:56.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:56 smithi031 ceph-mon[29457]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:56.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:56 smithi031 ceph-mon[29457]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:18:57.002 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-04-25T13:18:55.819264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652446,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":289,"num_read_kb":257,"num_write":258,"num_write_kb":3023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387644,"kb_used_data":13452,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554180,"statfs":{"total":1151940427776,"available":1151543480320,"internally_reserved":0,"allocated":13774848,"data_stored":7587030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003106"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948838+0000","last_change":"2024-04-25T13:17:40.825728+0000","last_active":"2024-04-25T13:18:32.948838+0000","last_peered":"2024-04-25T13:18:32.948838+0000","last_clean":"2024-04-25T13:18:32.948838+0000","last_became_active":"2024-04-25T13:17:40.824933+0000","last_became_peered":"2024-04-25T13:17:40.824933+0000","last_unstale":"2024-04-25T13:18:32.948838+0000","last_undegraded":"2024-04-25T13:18:32.948838+0000","last_fullsized":"2024-04-25T13:18:32.948838+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:14.326614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948553+0000","last_change":"2024-04-25T13:17:40.821148+0000","last_active":"2024-04-25T13:18:32.948553+0000","last_peered":"2024-04-25T13:18:32.948553+0000","last_clean":"2024-04-25T13:18:32.948553+0000","last_became_active":"2024-04-25T13:17:40.820244+0000","last_became_peered":"2024-04-25T13:17:40.820244+0000","last_unstale":"2024-04-25T13:18:32.948553+0000","last_undegraded":"2024-04-25T13:18:32.948553+0000","last_fullsized":"2024-04-25T13:18:32.948553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.566793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948485+0000","last_change":"2024-04-25T13:17:40.818729+0000","last_active":"2024-04-25T13:18:32.948485+0000","last_peered":"2024-04-25T13:18:32.948485+0000","last_clean":"2024-04-25T13:18:32.948485+0000","last_became_active":"2024-04-25T13:17:40.818351+0000","last_became_peered":"2024-04-25T13:17:40.818351+0000","last_unstale":"2024-04-25T13:18:32.948485+0000","last_undegraded":"2024-04-25T13:18:32.948485+0000","last_fullsized":"2024-04-25T13:18:32.948485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:42.412891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.289974+0000","last_change":"2024-04-25T13:17:40.820932+0000","last_active":"2024-04-25T13:18:33.289974+0000","last_peered":"2024-04-25T13:18:33.289974+0000","last_clean":"2024-04-25T13:18:33.289974+0000","last_became_active":"2024-04-25T13:17:40.820525+0000","last_became_peered":"2024-04-25T13:17:40.820525+0000","last_unstale":"2024-04-25T13:18:33.289974+0000","last_undegraded":"2024-04-25T13:18:33.289974+0000","last_fullsized":"2024-04-25T13:18:33.289974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:12.077703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949658+0000","last_change":"2024-04-25T13:17:40.829959+0000","last_active":"2024-04-25T13:18:32.949658+0000","last_peered":"2024-04-25T13:18:32.949658+0000","last_clean":"2024-04-25T13:18:32.949658+0000","last_became_active":"2024-04-25T13:17:40.829594+0000","last_became_peered":"2024-04-25T13:17:40.829594+0000","last_unstale":"2024-04-25T13:18:32.949658+0000","last_undegraded":"2024-04-25T13:18:32.949658+0000","last_fullsized":"2024-04-25T13:18:32.949658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:55.719296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332296+0000","last_change":"2024-04-25T13:17:40.817436+0000","last_active":"2024-04-25T13:18:33.332296+0000","last_peered":"2024-04-25T13:18:33.332296+0000","last_clean":"2024-04-25T13:18:33.332296+0000","last_became_active":"2024-04-25T13:17:40.816763+0000","last_became_peered":"2024-04-25T13:17:40.816763+0000","last_unstale":"2024-04-25T13:18:33.332296+0000","last_undegraded":"2024-04-25T13:18:33.332296+0000","last_fullsized":"2024-04-25T13:18:33.332296+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:09.505890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.166779+0000","last_change":"2024-04-25T13:17:40.822968+0000","last_active":"2024-04-25T13:18:33.166779+0000","last_peered":"2024-04-25T13:18:33.166779+0000","last_clean":"2024-04-25T13:18:33.166779+0000","last_became_active":"2024-04-25T13:17:40.822600+0000","last_became_peered":"2024-04-25T13:17:40.822600+0000","last_unstale":"2024-04-25T13:18:33.166779+0000","last_undegraded":"2024-04-25T13:18:33.166779+0000","last_fullsized":"2024-04-25T13:18:33.166779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:08:32.276310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189451+0000","last_change":"2024-04-25T13:17:40.823421+0000","last_active":"2024-04-25T13:18:33.189451+0000","last_peered":"2024-04-25T13:18:33.189451+0000","last_clean":"2024-04-25T13:18:33.189451+0000","last_became_active":"2024-04-25T13:17:40.823054+0000","last_became_peered":"2024-04-25T13:17:40.823054+0000","last_unstale":"2024-04-25T13:18:33.189451+0000","last_undegraded":"2024-04-25T13:18:33.189451+0000","last_fullsized":"2024-04-25T13:18:33.189451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:34.256763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.951026+0000","last_change":"2024-04-25T13:17:40.826105+0000","last_active":"2024-04-25T13:18:32.951026+0000","last_peered":"2024-04-25T13:18:32.951026+0000","last_clean":"2024-04-25T13:18:32.951026+0000","last_became_active":"2024-04-25T13:17:40.825763+0000","last_became_peered":"2024-04-25T13:17:40.825763+0000","last_unstale":"2024-04-25T13:18:32.951026+0000","last_undegraded":"2024-04-25T13:18:32.951026+0000","last_fullsized":"2024-04-25T13:18:32.951026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:07.058115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332705+0000","last_change":"2024-04-25T13:17:40.817356+0000","last_active":"2024-04-25T13:18:33.332705+0000","last_peered":"2024-04-25T13:18:33.332705+0000","last_clean":"2024-04-25T13:18:33.332705+0000","last_became_active":"2024-04-25T13:17:40.816906+0000","last_became_peered":"2024-04-25T13:17:40.816906+0000","last_unstale":"2024-04-25T13:18:33.332705+0000","last_undegraded":"2024-04-25T13:18:33.332705+0000","last_fullsized":"2024-04-25T13:18:33.332705+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167600+0000","last_change":"2024-04-25T13:17:40.829445+0000","last_active":"2024-04-25T13:18:33.167600+0000","last_peered":"2024-04-25T13:18:33.167600+0000","last_clean":"2024-04-25T13:18:33.167600+0000","last_became_active":"2024-04-25T13:17:40.826709+0000","last_became_peered":"2024-04-25T13:17:40.826709+0000","last_unstale":"2024-04-25T13:18:33.167600+0000","last_undegraded":"2024-04-25T13:18:33.167600+0000","last_fullsized":"2024-04-25T13:18:33.167600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:43.044192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.289827+0000","last_change":"2024-04-25T13:17:40.827486+0000","last_active":"2024-04-25T13:18:33.289827+0000","last_peered":"2024-04-25T13:18:33.289827+0000","last_clean":"2024-04-25T13:18:33.289827+0000","last_became_active":"2024-04-25T13:17:40.827070+0000","last_became_peered":"2024-04-25T13:17:40.827070+0000","last_unstale":"2024-04-25T13:18:33.289827+0000","last_undegraded":"2024-04-25T13:18:33.289827+0000","last_fullsized":"2024-04-25T13:18:33.289827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:39.564567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945336+0000","last_change":"2024-04-25T13:17:40.826645+0000","last_active":"2024-04-25T13:18:32.945336+0000","last_peered":"2024-04-25T13:18:32.945336+0000","last_clean":"2024-04-25T13:18:32.945336+0000","last_became_active":"2024-04-25T13:17:40.826314+0000","last_became_peered":"2024-04-25T13:17:40.826314+0000","last_unstale":"2024-04-25T13:18:32.945336+0000","last_undegraded":"2024-04-25T13:18:32.945336+0000","last_fullsized":"2024-04-25T13:18:32.945336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:59.600451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.947973+0000","last_change":"2024-04-25T13:17:40.814918+0000","last_active":"2024-04-25T13:18:32.947973+0000","last_peered":"2024-04-25T13:18:32.947973+0000","last_clean":"2024-04-25T13:18:32.947973+0000","last_became_active":"2024-04-25T13:17:40.814573+0000","last_became_peered":"2024-04-25T13:17:40.814573+0000","last_unstale":"2024-04-25T13:18:32.947973+0000","last_undegraded":"2024-04-25T13:18:32.947973+0000","last_fullsized":"2024-04-25T13:18:32.947973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:34.554390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.947805+0000","last_change":"2024-04-25T13:17:40.825481+0000","last_active":"2024-04-25T13:18:32.947805+0000","last_peered":"2024-04-25T13:18:32.947805+0000","last_clean":"2024-04-25T13:18:32.947805+0000","last_became_active":"2024-04-25T13:17:40.823726+0000","last_became_peered":"2024-04-25T13:17:40.823726+0000","last_unstale":"2024-04-25T13:18:32.947805+0000","last_undegraded":"2024-04-25T13:18:32.947805+0000","last_fullsized":"2024-04-25T13:18:32.947805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:46.500031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.950347+0000","last_change":"2024-04-25T13:17:40.827561+0000","last_active":"2024-04-25T13:18:32.950347+0000","last_peered":"2024-04-25T13:18:32.950347+0000","last_clean":"2024-04-25T13:18:32.950347+0000","last_became_active":"2024-04-25T13:17:40.827100+0000","last_became_peered":"2024-04-25T13:17:40.827100+0000","last_unstale":"2024-04-25T13:18:32.950347+0000","last_undegraded":"2024-04-25T13:18:32.950347+0000","last_fullsized":"2024-04-25T13:18:32.950347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:41.381469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949046+0000","last_change":"2024-04-25T13:17:40.826017+0000","last_active":"2024-04-25T13:18:32.949046+0000","last_peered":"2024-04-25T13:18:32.949046+0000","last_clean":"2024-04-25T13:18:32.949046+0000","last_became_active":"2024-04-25T13:17:40.825429+0000","last_became_peered":"2024-04-25T13:17:40.825429+0000","last_unstale":"2024-04-25T13:18:32.949046+0000","last_undegraded":"2024-04-25T13:18:32.949046+0000","last_fullsized":"2024-04-25T13:18:32.949046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:58.788260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.333588+0000","last_change":"2024-04-25T13:18:33.333588+0000","last_active":"2024-04-25T13:18:33.333588+0000","last_peered":"2024-04-25T13:18:33.333588+0000","last_clean":"2024-04-25T13:18:33.333588+0000","last_became_active":"2024-04-25T13:18:33.333310+0000","last_became_peered":"2024-04-25T13:18:33.333310+0000","last_unstale":"2024-04-25T13:18:33.333588+0000","last_undegraded":"2024-04-25T13:18:33.333588+0000","last_fullsized":"2024-04-25T13:18:33.333588+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:04.363715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964751+0000","last_change":"2024-04-25T13:17:40.824689+0000","last_active":"2024-04-25T13:18:32.964751+0000","last_peered":"2024-04-25T13:18:32.964751+0000","last_clean":"2024-04-25T13:18:32.964751+0000","last_became_active":"2024-04-25T13:17:40.823878+0000","last_became_peered":"2024-04-25T13:17:40.823878+0000","last_unstale":"2024-04-25T13:18:32.964751+0000","last_undegraded":"2024-04-25T13:18:32.964751+0000","last_fullsized":"2024-04-25T13:18:32.964751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:23.745510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948341+0000","last_change":"2024-04-25T13:17:39.812816+0000","last_active":"2024-04-25T13:18:32.948341+0000","last_peered":"2024-04-25T13:18:32.948341+0000","last_clean":"2024-04-25T13:18:32.948341+0000","last_became_active":"2024-04-25T13:17:39.812483+0000","last_became_peered":"2024-04-25T13:17:39.812483+0000","last_unstale":"2024-04-25T13:18:32.948341+0000","last_undegraded":"2024-04-25T13:18:32.948341+0000","last_fullsized":"2024-04-25T13:18:32.948341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:14.610763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189894+0000","last_change":"2024-04-25T13:17:40.824764+0000","last_active":"2024-04-25T13:18:33.189894+0000","last_peered":"2024-04-25T13:18:33.189894+0000","last_clean":"2024-04-25T13:18:33.189894+0000","last_became_active":"2024-04-25T13:17:40.824337+0000","last_became_peered":"2024-04-25T13:17:40.824337+0000","last_unstale":"2024-04-25T13:18:33.189894+0000","last_undegraded":"2024-04-25T13:18:33.189894+0000","last_fullsized":"2024-04-25T13:18:33.189894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:37.125246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.190058+0000","last_change":"2024-04-25T13:17:39.819820+0000","last_active":"2024-04-25T13:18:33.190058+0000","last_peered":"2024-04-25T13:18:33.190058+0000","last_clean":"2024-04-25T13:18:33.190058+0000","last_became_active":"2024-04-25T13:17:39.819446+0000","last_became_peered":"2024-04-25T13:17:39.819446+0000","last_unstale":"2024-04-25T13:18:33.190058+0000","last_undegraded":"2024-04-25T13:18:33.190058+0000","last_fullsized":"2024-04-25T13:18:33.190058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:36.902011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948781+0000","last_change":"2024-04-25T13:17:40.821183+0000","last_active":"2024-04-25T13:18:32.948781+0000","last_peered":"2024-04-25T13:18:32.948781+0000","last_clean":"2024-04-25T13:18:32.948781+0000","last_became_active":"2024-04-25T13:17:40.820789+0000","last_became_peered":"2024-04-25T13:17:40.820789+0000","last_unstale":"2024-04-25T13:18:32.948781+0000","last_undegraded":"2024-04-25T13:18:32.948781+0000","last_fullsized":"2024-04-25T13:18:32.948781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:56.225385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949888+0000","last_change":"2024-04-25T13:17:39.813132+0000","last_active":"2024-04-25T13:18:32.949888+0000","last_peered":"2024-04-25T13:18:32.949888+0000","last_clean":"2024-04-25T13:18:32.949888+0000","last_became_active":"2024-04-25T13:17:39.812837+0000","last_became_peered":"2024-04-25T13:17:39.812837+0000","last_unstale":"2024-04-25T13:18:32.949888+0000","last_undegraded":"2024-04-25T13:18:32.949888+0000","last_fullsized":"2024-04-25T13:18:32.949888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:55.342376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.950609+0000","last_change":"2024-04-25T13:17:40.824143+0000","last_active":"2024-04-25T13:18:32.950609+0000","last_peered":"2024-04-25T13:18:32.950609+0000","last_clean":"2024-04-25T13:18:32.950609+0000","last_became_active":"2024-04-25T13:17:40.823778+0000","last_became_peered":"2024-04-25T13:17:40.823778+0000","last_unstale":"2024-04-25T13:18:32.950609+0000","last_undegraded":"2024-04-25T13:18:32.950609+0000","last_fullsized":"2024-04-25T13:18:32.950609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:54.112957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949534+0000","last_change":"2024-04-25T13:17:39.817743+0000","last_active":"2024-04-25T13:18:32.949534+0000","last_peered":"2024-04-25T13:18:32.949534+0000","last_clean":"2024-04-25T13:18:32.949534+0000","last_became_active":"2024-04-25T13:17:39.817392+0000","last_became_peered":"2024-04-25T13:17:39.817392+0000","last_unstale":"2024-04-25T13:18:32.949534+0000","last_undegraded":"2024-04-25T13:18:32.949534+0000","last_fullsized":"2024-04-25T13:18:32.949534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:16.400644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.946435+0000","last_change":"2024-04-25T13:17:40.829895+0000","last_active":"2024-04-25T13:18:32.946435+0000","last_peered":"2024-04-25T13:18:32.946435+0000","last_clean":"2024-04-25T13:18:32.946435+0000","last_became_active":"2024-04-25T13:17:40.829300+0000","last_became_peered":"2024-04-25T13:17:40.829300+0000","last_unstale":"2024-04-25T13:18:32.946435+0000","last_undegraded":"2024-04-25T13:18:32.946435+0000","last_fullsized":"2024-04-25T13:18:32.946435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:49.935081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.946196+0000","last_change":"2024-04-25T13:17:39.813231+0000","last_active":"2024-04-25T13:18:32.946196+0000","last_peered":"2024-04-25T13:18:32.946196+0000","last_clean":"2024-04-25T13:18:32.946196+0000","last_became_active":"2024-04-25T13:17:39.812441+0000","last_became_peered":"2024-04-25T13:17:39.812441+0000","last_unstale":"2024-04-25T13:18:32.946196+0000","last_undegraded":"2024-04-25T13:18:32.946196+0000","last_fullsized":"2024-04-25T13:18:32.946196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:17.578986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.168914+0000","last_change":"2024-04-25T13:18:33.168914+0000","last_active":"2024-04-25T13:18:33.168914+0000","last_peered":"2024-04-25T13:18:33.168914+0000","last_clean":"2024-04-25T13:18:33.168914+0000","last_became_active":"2024-04-25T13:18:33.168620+0000","last_became_peered":"2024-04-25T13:18:33.168620+0000","last_unstale":"2024-04-25T13:18:33.168914+0000","last_undegraded":"2024-04-25T13:18:33.168914+0000","last_fullsized":"2024-04-25T13:18:33.168914+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:03.478414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332193+0000","last_change":"2024-04-25T13:17:39.809888+0000","last_active":"2024-04-25T13:18:33.332193+0000","last_peered":"2024-04-25T13:18:33.332193+0000","last_clean":"2024-04-25T13:18:33.332193+0000","last_became_active":"2024-04-25T13:17:39.809519+0000","last_became_peered":"2024-04-25T13:17:39.809519+0000","last_unstale":"2024-04-25T13:18:33.332193+0000","last_undegraded":"2024-04-25T13:18:33.332193+0000","last_fullsized":"2024-04-25T13:18:33.332193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:43.306668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.190931+0000","last_change":"2024-04-25T13:18:33.190931+0000","last_active":"2024-04-25T13:18:33.190931+0000","last_peered":"2024-04-25T13:18:33.190931+0000","last_clean":"2024-04-25T13:18:33.190931+0000","last_became_active":"2024-04-25T13:18:33.190517+0000","last_became_peered":"2024-04-25T13:18:33.190517+0000","last_unstale":"2024-04-25T13:18:33.190931+0000","last_undegraded":"2024-04-25T13:18:33.190931+0000","last_fullsized":"2024-04-25T13:18:33.190931+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:32.925276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945288+0000","last_change":"2024-04-25T13:17:39.811231+0000","last_active":"2024-04-25T13:18:32.945288+0000","last_peered":"2024-04-25T13:18:32.945288+0000","last_clean":"2024-04-25T13:18:32.945288+0000","last_became_active":"2024-04-25T13:17:39.810762+0000","last_became_peered":"2024-04-25T13:17:39.810762+0000","last_unstale":"2024-04-25T13:18:32.945288+0000","last_undegraded":"2024-04-25T13:18:32.945288+0000","last_fullsized":"2024-04-25T13:18:32.945288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:15.970042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189536+0000","last_change":"2024-04-25T13:17:40.828074+0000","last_active":"2024-04-25T13:18:33.189536+0000","last_peered":"2024-04-25T13:18:33.189536+0000","last_clean":"2024-04-25T13:18:33.189536+0000","last_became_active":"2024-04-25T13:17:40.827702+0000","last_became_peered":"2024-04-25T13:17:40.827702+0000","last_unstale":"2024-04-25T13:18:33.189536+0000","last_undegraded":"2024-04-25T13:18:33.189536+0000","last_fullsized":"2024-04-25T13:18:33.189536+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:23.408908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945918+0000","last_change":"2024-04-25T13:17:39.812746+0000","last_active":"2024-04-25T13:18:32.945918+0000","last_peered":"2024-04-25T13:18:32.945918+0000","last_clean":"2024-04-25T13:18:32.945918+0000","last_became_active":"2024-04-25T13:17:39.812163+0000","last_became_peered":"2024-04-25T13:17:39.812163+0000","last_unstale":"2024-04-25T13:18:32.945918+0000","last_undegraded":"2024-04-25T13:18:32.945918+0000","last_fullsized":"2024-04-25T13:18:32.945918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:20.410328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948238+0000","last_change":"2024-04-25T13:17:40.829512+0000","last_active":"2024-04-25T13:18:32.948238+0000","last_peered":"2024-04-25T13:18:32.948238+0000","last_clean":"2024-04-25T13:18:32.948238+0000","last_became_active":"2024-04-25T13:17:40.829084+0000","last_became_peered":"2024-04-25T13:17:40.829084+0000","last_unstale":"2024-04-25T13:18:32.948238+0000","last_undegraded":"2024-04-25T13:18:32.948238+0000","last_fullsized":"2024-04-25T13:18:32.948238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:55.473018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332622+0000","last_change":"2024-04-25T13:17:39.810766+0000","last_active":"2024-04-25T13:18:33.332622+0000","last_peered":"2024-04-25T13:18:33.332622+0000","last_clean":"2024-04-25T13:18:33.332622+0000","last_became_active":"2024-04-25T13:17:39.810444+0000","last_became_peered":"2024-04-25T13:17:39.810444+0000","last_unstale":"2024-04-25T13:18:33.332622+0000","last_undegraded":"2024-04-25T13:18:33.332622+0000","last_fullsized":"2024-04-25T13:18:33.332622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:33.470819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.950846+0000","last_change":"2024-04-25T13:17:40.825574+0000","last_active":"2024-04-25T13:18:32.950846+0000","last_peered":"2024-04-25T13:18:32.950846+0000","last_clean":"2024-04-25T13:18:32.950846+0000","last_became_active":"2024-04-25T13:17:40.825220+0000","last_became_peered":"2024-04-25T13:17:40.825220+0000","last_unstale":"2024-04-25T13:18:32.950846+0000","last_undegraded":"2024-04-25T13:18:32.950846+0000","last_fullsized":"2024-04-25T13:18:32.950846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:13.941665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167496+0000","last_change":"2024-04-25T13:17:39.816040+0000","last_active":"2024-04-25T13:18:33.167496+0000","last_peered":"2024-04-25T13:18:33.167496+0000","last_clean":"2024-04-25T13:18:33.167496+0000","last_became_active":"2024-04-25T13:17:39.815746+0000","last_became_peered":"2024-04-25T13:17:39.815746+0000","last_unstale":"2024-04-25T13:18:33.167496+0000","last_undegraded":"2024-04-25T13:18:33.167496+0000","last_fullsized":"2024-04-25T13:18:33.167496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:12.068992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948704+0000","last_change":"2024-04-25T13:17:40.821926+0000","last_active":"2024-04-25T13:18:32.948704+0000","last_peered":"2024-04-25T13:18:32.948704+0000","last_clean":"2024-04-25T13:18:32.948704+0000","last_became_active":"2024-04-25T13:17:40.821280+0000","last_became_peered":"2024-04-25T13:17:40.821280+0000","last_unstale":"2024-04-25T13:18:32.948704+0000","last_undegraded":"2024-04-25T13:18:32.948704+0000","last_fullsized":"2024-04-25T13:18:32.948704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:33.662553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'3","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.946542+0000","last_change":"2024-04-25T13:17:39.819113+0000","last_active":"2024-04-25T13:18:32.946542+0000","last_peered":"2024-04-25T13:18:32.946542+0000","last_clean":"2024-04-25T13:18:32.946542+0000","last_became_active":"2024-04-25T13:17:39.818749+0000","last_became_peered":"2024-04-25T13:17:39.818749+0000","last_unstale":"2024-04-25T13:18:32.946542+0000","last_undegraded":"2024-04-25T13:18:32.946542+0000","last_fullsized":"2024-04-25T13:18:32.946542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:32.119616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.950657+0000","last_change":"2024-04-25T13:17:40.821066+0000","last_active":"2024-04-25T13:18:32.950657+0000","last_peered":"2024-04-25T13:18:32.950657+0000","last_clean":"2024-04-25T13:18:32.950657+0000","last_became_active":"2024-04-25T13:17:40.820674+0000","last_became_peered":"2024-04-25T13:17:40.820674+0000","last_unstale":"2024-04-25T13:18:32.950657+0000","last_undegraded":"2024-04-25T13:18:32.950657+0000","last_fullsized":"2024-04-25T13:18:32.950657+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:01.157850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":192,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:52.946145+0000","last_change":"2024-04-25T13:17:39.811131+0000","last_active":"2024-04-25T13:18:52.946145+0000","last_peered":"2024-04-25T13:18:52.946145+0000","last_clean":"2024-04-25T13:18:52.946145+0000","last_became_active":"2024-04-25T13:17:39.810791+0000","last_became_peered":"2024-04-25T13:17:39.810791+0000","last_unstale":"2024-04-25T13:18:52.946145+0000","last_undegraded":"2024-04-25T13:18:52.946145+0000","last_fullsized":"2024-04-25T13:18:52.946145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:43.964183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":147,"num_read_kb":132,"num_write":26,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949577+0000","last_change":"2024-04-25T13:17:40.818077+0000","last_active":"2024-04-25T13:18:32.949577+0000","last_peered":"2024-04-25T13:18:32.949577+0000","last_clean":"2024-04-25T13:18:32.949577+0000","last_became_active":"2024-04-25T13:17:40.817757+0000","last_became_peered":"2024-04-25T13:17:40.817757+0000","last_unstale":"2024-04-25T13:18:32.949577+0000","last_undegraded":"2024-04-25T13:18:32.949577+0000","last_fullsized":"2024-04-25T13:18:32.949577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:17.526603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964479+0000","last_change":"2024-04-25T13:17:39.809111+0000","last_active":"2024-04-25T13:18:32.964479+0000","last_peered":"2024-04-25T13:18:32.964479+0000","last_clean":"2024-04-25T13:18:32.964479+0000","last_became_active":"2024-04-25T13:17:39.808785+0000","last_became_peered":"2024-04-25T13:17:39.808785+0000","last_unstale":"2024-04-25T13:18:32.964479+0000","last_undegraded":"2024-04-25T13:18:32.964479+0000","last_fullsized":"2024-04-25T13:18:32.964479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:41.549940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945334+0000","last_change":"2024-04-25T13:17:40.835646+0000","last_active":"2024-04-25T13:18:32.945334+0000","last_peered":"2024-04-25T13:18:32.945334+0000","last_clean":"2024-04-25T13:18:32.945334+0000","last_became_active":"2024-04-25T13:17:40.835267+0000","last_became_peered":"2024-04-25T13:17:40.835267+0000","last_unstale":"2024-04-25T13:18:32.945334+0000","last_undegraded":"2024-04-25T13:18:32.945334+0000","last_fullsized":"2024-04-25T13:18:32.945334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:05.811651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.946146+0000","last_change":"2024-04-25T13:17:39.813006+0000","last_active":"2024-04-25T13:18:32.946146+0000","last_peered":"2024-04-25T13:18:32.946146+0000","last_clean":"2024-04-25T13:18:32.946146+0000","last_became_active":"2024-04-25T13:17:39.812635+0000","last_became_peered":"2024-04-25T13:17:39.812635+0000","last_unstale":"2024-04-25T13:18:32.946146+0000","last_undegraded":"2024-04-25T13:18:32.946146+0000","last_fullsized":"2024-04-25T13:18:32.946146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:59.922568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167291+0000","last_change":"2024-04-25T13:17:40.833365+0000","last_active":"2024-04-25T13:18:33.167291+0000","last_peered":"2024-04-25T13:18:33.167291+0000","last_clean":"2024-04-25T13:18:33.167291+0000","last_became_active":"2024-04-25T13:17:40.832927+0000","last_became_peered":"2024-04-25T13:17:40.832927+0000","last_unstale":"2024-04-25T13:18:33.167291+0000","last_undegraded":"2024-04-25T13:18:33.167291+0000","last_fullsized":"2024-04-25T13:18:33.167291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.622153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.289871+0000","last_change":"2024-04-25T13:17:39.810983+0000","last_active":"2024-04-25T13:18:33.289871+0000","last_peered":"2024-04-25T13:18:33.289871+0000","last_clean":"2024-04-25T13:18:33.289871+0000","last_became_active":"2024-04-25T13:17:39.810366+0000","last_became_peered":"2024-04-25T13:17:39.810366+0000","last_unstale":"2024-04-25T13:18:33.289871+0000","last_undegraded":"2024-04-25T13:18:33.289871+0000","last_fullsized":"2024-04-25T13:18:33.289871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:33.636924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.166706+0000","last_change":"2024-04-25T13:17:40.818301+0000","last_active":"2024-04-25T13:18:33.166706+0000","last_peered":"2024-04-25T13:18:33.166706+0000","last_clean":"2024-04-25T13:18:33.166706+0000","last_became_active":"2024-04-25T13:17:40.817903+0000","last_became_peered":"2024-04-25T13:17:40.817903+0000","last_unstale":"2024-04-25T13:18:33.166706+0000","last_undegraded":"2024-04-25T13:18:33.166706+0000","last_fullsized":"2024-04-25T13:18:33.166706+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:40.913771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948966+0000","last_change":"2024-04-25T13:17:39.820411+0000","last_active":"2024-04-25T13:18:32.948966+0000","last_peered":"2024-04-25T13:18:32.948966+0000","last_clean":"2024-04-25T13:18:32.948966+0000","last_became_active":"2024-04-25T13:17:39.820051+0000","last_became_peered":"2024-04-25T13:17:39.820051+0000","last_unstale":"2024-04-25T13:18:32.948966+0000","last_undegraded":"2024-04-25T13:18:32.948966+0000","last_fullsized":"2024-04-25T13:18:32.948966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:08.931024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.291455+0000","last_change":"2024-04-25T13:18:33.291044+0000","last_active":"2024-04-25T13:18:33.291455+0000","last_peered":"2024-04-25T13:18:33.291455+0000","last_clean":"2024-04-25T13:18:33.291455+0000","last_became_active":"2024-04-25T13:18:33.290673+0000","last_became_peered":"2024-04-25T13:18:33.290673+0000","last_unstale":"2024-04-25T13:18:33.291455+0000","last_undegraded":"2024-04-25T13:18:33.291455+0000","last_fullsized":"2024-04-25T13:18:33.291455+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:41.657993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949901+0000","last_change":"2024-04-25T13:17:39.812160+0000","last_active":"2024-04-25T13:18:32.949901+0000","last_peered":"2024-04-25T13:18:32.949901+0000","last_clean":"2024-04-25T13:18:32.949901+0000","last_became_active":"2024-04-25T13:17:39.811846+0000","last_became_peered":"2024-04-25T13:17:39.811846+0000","last_unstale":"2024-04-25T13:18:32.949901+0000","last_undegraded":"2024-04-25T13:18:32.949901+0000","last_fullsized":"2024-04-25T13:18:32.949901+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:20.146245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.951554+0000","last_change":"2024-04-25T13:18:32.951554+0000","last_active":"2024-04-25T13:18:32.951554+0000","last_peered":"2024-04-25T13:18:32.951554+0000","last_clean":"2024-04-25T13:18:32.951554+0000","last_became_active":"2024-04-25T13:18:32.951082+0000","last_became_peered":"2024-04-25T13:18:32.951082+0000","last_unstale":"2024-04-25T13:18:32.951554+0000","last_undegraded":"2024-04-25T13:18:32.951554+0000","last_fullsized":"2024-04-25T13:18:32.951554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:41.550497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948192+0000","last_change":"2024-04-25T13:17:39.819540+0000","last_active":"2024-04-25T13:18:32.948192+0000","last_peered":"2024-04-25T13:18:32.948192+0000","last_clean":"2024-04-25T13:18:32.948192+0000","last_became_active":"2024-04-25T13:17:39.818942+0000","last_became_peered":"2024-04-25T13:17:39.818942+0000","last_unstale":"2024-04-25T13:18:32.948192+0000","last_undegraded":"2024-04-25T13:18:32.948192+0000","last_fullsized":"2024-04-25T13:18:32.948192+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:51.300087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332613+0000","last_change":"2024-04-25T13:17:40.818872+0000","last_active":"2024-04-25T13:18:33.332613+0000","last_peered":"2024-04-25T13:18:33.332613+0000","last_clean":"2024-04-25T13:18:33.332613+0000","last_became_active":"2024-04-25T13:17:40.818522+0000","last_became_peered":"2024-04-25T13:17:40.818522+0000","last_unstale":"2024-04-25T13:18:33.332613+0000","last_undegraded":"2024-04-25T13:18:33.332613+0000","last_fullsized":"2024-04-25T13:18:33.332613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.928782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167703+0000","last_change":"2024-04-25T13:17:39.815555+0000","last_active":"2024-04-25T13:18:33.167703+0000","last_peered":"2024-04-25T13:18:33.167703+0000","last_clean":"2024-04-25T13:18:33.167703+0000","last_became_active":"2024-04-25T13:17:39.815249+0000","last_became_peered":"2024-04-25T13:17:39.815249+0000","last_unstale":"2024-04-25T13:18:33.167703+0000","last_undegraded":"2024-04-25T13:18:33.167703+0000","last_fullsized":"2024-04-25T13:18:33.167703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:40.723562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.332505+0000","last_change":"2024-04-25T13:17:40.821643+0000","last_active":"2024-04-25T13:18:33.332505+0000","last_peered":"2024-04-25T13:18:33.332505+0000","last_clean":"2024-04-25T13:18:33.332505+0000","last_became_active":"2024-04-25T13:17:40.820715+0000","last_became_peered":"2024-04-25T13:17:40.820715+0000","last_unstale":"2024-04-25T13:18:33.332505+0000","last_undegraded":"2024-04-25T13:18:33.332505+0000","last_fullsized":"2024-04-25T13:18:33.332505+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:54.046030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.965993+0000","last_change":"2024-04-25T13:18:32.965993+0000","last_active":"2024-04-25T13:18:32.965993+0000","last_peered":"2024-04-25T13:18:32.965993+0000","last_clean":"2024-04-25T13:18:32.965993+0000","last_became_active":"2024-04-25T13:18:32.965546+0000","last_became_peered":"2024-04-25T13:18:32.965546+0000","last_unstale":"2024-04-25T13:18:32.965993+0000","last_undegraded":"2024-04-25T13:18:32.965993+0000","last_fullsized":"2024-04-25T13:18:32.965993+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:10.496431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945162+0000","last_change":"2024-04-25T13:17:39.820658+0000","last_active":"2024-04-25T13:18:32.945162+0000","last_peered":"2024-04-25T13:18:32.945162+0000","last_clean":"2024-04-25T13:18:32.945162+0000","last_became_active":"2024-04-25T13:17:39.820122+0000","last_became_peered":"2024-04-25T13:17:39.820122+0000","last_unstale":"2024-04-25T13:18:32.945162+0000","last_undegraded":"2024-04-25T13:18:32.945162+0000","last_fullsized":"2024-04-25T13:18:32.945162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:04.655982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189662+0000","last_change":"2024-04-25T13:17:40.827220+0000","last_active":"2024-04-25T13:18:33.189662+0000","last_peered":"2024-04-25T13:18:33.189662+0000","last_clean":"2024-04-25T13:18:33.189662+0000","last_became_active":"2024-04-25T13:17:40.826517+0000","last_became_peered":"2024-04-25T13:17:40.826517+0000","last_unstale":"2024-04-25T13:18:33.189662+0000","last_undegraded":"2024-04-25T13:18:33.189662+0000","last_fullsized":"2024-04-25T13:18:33.189662+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:30.405644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.289527+0000","last_change":"2024-04-25T13:17:39.813893+0000","last_active":"2024-04-25T13:18:33.289527+0000","last_peered":"2024-04-25T13:18:33.289527+0000","last_clean":"2024-04-25T13:18:33.289527+0000","last_became_active":"2024-04-25T13:17:39.812939+0000","last_became_peered":"2024-04-25T13:17:39.812939+0000","last_unstale":"2024-04-25T13:18:33.289527+0000","last_undegraded":"2024-04-25T13:18:33.289527+0000","last_fullsized":"2024-04-25T13:18:33.289527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.033913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.166824+0000","last_change":"2024-04-25T13:16:30.734601+0000","last_active":"2024-04-25T13:18:33.166824+0000","last_peered":"2024-04-25T13:18:33.166824+0000","last_clean":"2024-04-25T13:18:33.166824+0000","last_became_active":"2024-04-25T13:16:30.733987+0000","last_became_peered":"2024-04-25T13:16:30.733987+0000","last_unstale":"2024-04-25T13:18:33.166824+0000","last_undegraded":"2024-04-25T13:18:33.166824+0000","last_fullsized":"2024-04-25T13:18:33.166824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:13:02.186391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:13:02.186391+0000","last_clean_scrub_stamp":"2024-04-25T13:13:02.186391+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:41.669205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.965694+0000","last_change":"2024-04-25T13:18:32.965694+0000","last_active":"2024-04-25T13:18:32.965694+0000","last_peered":"2024-04-25T13:18:32.965694+0000","last_clean":"2024-04-25T13:18:32.965694+0000","last_became_active":"2024-04-25T13:18:32.965279+0000","last_became_peered":"2024-04-25T13:18:32.965279+0000","last_unstale":"2024-04-25T13:18:32.965694+0000","last_undegraded":"2024-04-25T13:18:32.965694+0000","last_fullsized":"2024-04-25T13:18:32.965694+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:28.962446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.166943+0000","last_change":"2024-04-25T13:17:39.809530+0000","last_active":"2024-04-25T13:18:33.166943+0000","last_peered":"2024-04-25T13:18:33.166943+0000","last_clean":"2024-04-25T13:18:33.166943+0000","last_became_active":"2024-04-25T13:17:39.809198+0000","last_became_peered":"2024-04-25T13:17:39.809198+0000","last_unstale":"2024-04-25T13:18:33.166943+0000","last_undegraded":"2024-04-25T13:18:33.166943+0000","last_fullsized":"2024-04-25T13:18:33.166943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:34.397912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949700+0000","last_change":"2024-04-25T13:17:39.819607+0000","last_active":"2024-04-25T13:18:32.949700+0000","last_peered":"2024-04-25T13:18:32.949700+0000","last_clean":"2024-04-25T13:18:32.949700+0000","last_became_active":"2024-04-25T13:17:39.819273+0000","last_became_peered":"2024-04-25T13:17:39.819273+0000","last_unstale":"2024-04-25T13:18:32.949700+0000","last_undegraded":"2024-04-25T13:18:32.949700+0000","last_fullsized":"2024-04-25T13:18:32.949700+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:39.414085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945248+0000","last_change":"2024-04-25T13:17:40.824554+0000","last_active":"2024-04-25T13:18:32.945248+0000","last_peered":"2024-04-25T13:18:32.945248+0000","last_clean":"2024-04-25T13:18:32.945248+0000","last_became_active":"2024-04-25T13:17:40.824249+0000","last_became_peered":"2024-04-25T13:17:40.824249+0000","last_unstale":"2024-04-25T13:18:32.945248+0000","last_undegraded":"2024-04-25T13:18:32.945248+0000","last_fullsized":"2024-04-25T13:18:32.945248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:17.282312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948329+0000","last_change":"2024-04-25T13:17:39.820095+0000","last_active":"2024-04-25T13:18:32.948329+0000","last_peered":"2024-04-25T13:18:32.948329+0000","last_clean":"2024-04-25T13:18:32.948329+0000","last_became_active":"2024-04-25T13:17:39.819774+0000","last_became_peered":"2024-04-25T13:17:39.819774+0000","last_unstale":"2024-04-25T13:18:32.948329+0000","last_undegraded":"2024-04-25T13:18:32.948329+0000","last_fullsized":"2024-04-25T13:18:32.948329+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:09.154958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189963+0000","last_change":"2024-04-25T13:17:40.830202+0000","last_active":"2024-04-25T13:18:33.189963+0000","last_peered":"2024-04-25T13:18:33.189963+0000","last_clean":"2024-04-25T13:18:33.189963+0000","last_became_active":"2024-04-25T13:17:40.829814+0000","last_became_peered":"2024-04-25T13:17:40.829814+0000","last_unstale":"2024-04-25T13:18:33.189963+0000","last_undegraded":"2024-04-25T13:18:33.189963+0000","last_fullsized":"2024-04-25T13:18:33.189963+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:05.492378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.289947+0000","last_change":"2024-04-25T13:17:39.814680+0000","last_active":"2024-04-25T13:18:33.289947+0000","last_peered":"2024-04-25T13:18:33.289947+0000","last_clean":"2024-04-25T13:18:33.289947+0000","last_became_active":"2024-04-25T13:17:39.814324+0000","last_became_peered":"2024-04-25T13:17:39.814324+0000","last_unstale":"2024-04-25T13:18:33.289947+0000","last_undegraded":"2024-04-25T13:18:33.289947+0000","last_fullsized":"2024-04-25T13:18:33.289947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:42.818060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.290964+0000","last_change":"2024-04-25T13:18:33.290964+0000","last_active":"2024-04-25T13:18:33.290964+0000","last_peered":"2024-04-25T13:18:33.290964+0000","last_clean":"2024-04-25T13:18:33.290964+0000","last_became_active":"2024-04-25T13:18:33.290420+0000","last_became_peered":"2024-04-25T13:18:33.290420+0000","last_unstale":"2024-04-25T13:18:33.290964+0000","last_undegraded":"2024-04-25T13:18:33.290964+0000","last_fullsized":"2024-04-25T13:18:33.290964+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:34.696247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.950358+0000","last_change":"2024-04-25T13:17:39.820908+0000","last_active":"2024-04-25T13:18:32.950358+0000","last_peered":"2024-04-25T13:18:32.950358+0000","last_clean":"2024-04-25T13:18:32.950358+0000","last_became_active":"2024-04-25T13:17:39.820612+0000","last_became_peered":"2024-04-25T13:17:39.820612+0000","last_unstale":"2024-04-25T13:18:32.950358+0000","last_undegraded":"2024-04-25T13:18:32.950358+0000","last_fullsized":"2024-04-25T13:18:32.950358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:15.170079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948951+0000","last_change":"2024-04-25T13:17:40.817232+0000","last_active":"2024-04-25T13:18:32.948951+0000","last_peered":"2024-04-25T13:18:32.948951+0000","last_clean":"2024-04-25T13:18:32.948951+0000","last_became_active":"2024-04-25T13:17:40.816895+0000","last_became_peered":"2024-04-25T13:17:40.816895+0000","last_unstale":"2024-04-25T13:18:32.948951+0000","last_undegraded":"2024-04-25T13:18:32.948951+0000","last_fullsized":"2024-04-25T13:18:32.948951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:28.772948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189546+0000","last_change":"2024-04-25T13:17:39.816586+0000","last_active":"2024-04-25T13:18:33.189546+0000","last_peered":"2024-04-25T13:18:33.189546+0000","last_clean":"2024-04-25T13:18:33.189546+0000","last_became_active":"2024-04-25T13:17:39.815712+0000","last_became_peered":"2024-04-25T13:17:39.815712+0000","last_unstale":"2024-04-25T13:18:33.189546+0000","last_undegraded":"2024-04-25T13:18:33.189546+0000","last_fullsized":"2024-04-25T13:18:33.189546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.772789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948386+0000","last_change":"2024-04-25T13:17:40.824505+0000","last_active":"2024-04-25T13:18:32.948386+0000","last_peered":"2024-04-25T13:18:32.948386+0000","last_clean":"2024-04-25T13:18:32.948386+0000","last_became_active":"2024-04-25T13:17:40.824197+0000","last_became_peered":"2024-04-25T13:17:40.824197+0000","last_unstale":"2024-04-25T13:18:32.948386+0000","last_undegraded":"2024-04-25T13:18:32.948386+0000","last_fullsized":"2024-04-25T13:18:32.948386+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:17.484269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964352+0000","last_change":"2024-04-25T13:17:39.810242+0000","last_active":"2024-04-25T13:18:32.964352+0000","last_peered":"2024-04-25T13:18:32.964352+0000","last_clean":"2024-04-25T13:18:32.964352+0000","last_became_active":"2024-04-25T13:17:39.809547+0000","last_became_peered":"2024-04-25T13:17:39.809547+0000","last_unstale":"2024-04-25T13:18:32.964352+0000","last_undegraded":"2024-04-25T13:18:32.964352+0000","last_fullsized":"2024-04-25T13:18:32.964352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.615321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189756+0000","last_change":"2024-04-25T13:17:40.822114+0000","last_active":"2024-04-25T13:18:33.189756+0000","last_peered":"2024-04-25T13:18:33.189756+0000","last_clean":"2024-04-25T13:18:33.189756+0000","last_became_active":"2024-04-25T13:17:40.821517+0000","last_became_peered":"2024-04-25T13:17:40.821517+0000","last_unstale":"2024-04-25T13:18:33.189756+0000","last_undegraded":"2024-04-25T13:18:33.189756+0000","last_fullsized":"2024-04-25T13:18:33.189756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:57:07.225119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949571+0000","last_change":"2024-04-25T13:17:39.812135+0000","last_active":"2024-04-25T13:18:32.949571+0000","last_peered":"2024-04-25T13:18:32.949571+0000","last_clean":"2024-04-25T13:18:32.949571+0000","last_became_active":"2024-04-25T13:17:39.811836+0000","last_became_peered":"2024-04-25T13:17:39.811836+0000","last_unstale":"2024-04-25T13:18:32.949571+0000","last_undegraded":"2024-04-25T13:18:32.949571+0000","last_fullsized":"2024-04-25T13:18:32.949571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:00.692725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167708+0000","last_change":"2024-04-25T13:17:40.830160+0000","last_active":"2024-04-25T13:18:33.167708+0000","last_peered":"2024-04-25T13:18:33.167708+0000","last_clean":"2024-04-25T13:18:33.167708+0000","last_became_active":"2024-04-25T13:17:40.829773+0000","last_became_peered":"2024-04-25T13:17:40.829773+0000","last_unstale":"2024-04-25T13:18:33.167708+0000","last_undegraded":"2024-04-25T13:18:33.167708+0000","last_fullsized":"2024-04-25T13:18:33.167708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:24.515333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964731+0000","last_change":"2024-04-25T13:17:39.819011+0000","last_active":"2024-04-25T13:18:32.964731+0000","last_peered":"2024-04-25T13:18:32.964731+0000","last_clean":"2024-04-25T13:18:32.964731+0000","last_became_active":"2024-04-25T13:17:39.818656+0000","last_became_peered":"2024-04-25T13:17:39.818656+0000","last_unstale":"2024-04-25T13:18:32.964731+0000","last_undegraded":"2024-04-25T13:18:32.964731+0000","last_fullsized":"2024-04-25T13:18:32.964731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:20.507500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167790+0000","last_change":"2024-04-25T13:17:40.829595+0000","last_active":"2024-04-25T13:18:33.167790+0000","last_peered":"2024-04-25T13:18:33.167790+0000","last_clean":"2024-04-25T13:18:33.167790+0000","last_became_active":"2024-04-25T13:17:40.827707+0000","last_became_peered":"2024-04-25T13:17:40.827707+0000","last_unstale":"2024-04-25T13:18:33.167790+0000","last_undegraded":"2024-04-25T13:18:33.167790+0000","last_fullsized":"2024-04-25T13:18:33.167790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:49.002629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.946543+0000","last_change":"2024-04-25T13:17:39.809740+0000","last_active":"2024-04-25T13:18:32.946543+0000","last_peered":"2024-04-25T13:18:32.946543+0000","last_clean":"2024-04-25T13:18:32.946543+0000","last_became_active":"2024-04-25T13:17:39.809407+0000","last_became_peered":"2024-04-25T13:17:39.809407+0000","last_unstale":"2024-04-25T13:18:32.946543+0000","last_undegraded":"2024-04-25T13:18:32.946543+0000","last_fullsized":"2024-04-25T13:18:32.946543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:16.732755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167519+0000","last_change":"2024-04-25T13:17:40.825573+0000","last_active":"2024-04-25T13:18:33.167519+0000","last_peered":"2024-04-25T13:18:33.167519+0000","last_clean":"2024-04-25T13:18:33.167519+0000","last_became_active":"2024-04-25T13:17:40.825181+0000","last_became_peered":"2024-04-25T13:17:40.825181+0000","last_unstale":"2024-04-25T13:18:33.167519+0000","last_undegraded":"2024-04-25T13:18:33.167519+0000","last_fullsized":"2024-04-25T13:18:33.167519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:06.446494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945434+0000","last_change":"2024-04-25T13:17:39.812458+0000","last_active":"2024-04-25T13:18:32.945434+0000","last_peered":"2024-04-25T13:18:32.945434+0000","last_clean":"2024-04-25T13:18:32.945434+0000","last_became_active":"2024-04-25T13:17:39.812092+0000","last_became_peered":"2024-04-25T13:17:39.812092+0000","last_unstale":"2024-04-25T13:18:32.945434+0000","last_undegraded":"2024-04-25T13:18:32.945434+0000","last_fullsized":"2024-04-25T13:18:32.945434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:38.783139+0000","last_clean_scrub_stamp":"2024-04-25T13:17:38.783139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:07.210444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.167636+0000","last_change":"2024-04-25T13:17:40.821510+0000","last_active":"2024-04-25T13:18:33.167636+0000","last_peered":"2024-04-25T13:18:33.167636+0000","last_clean":"2024-04-25T13:18:33.167636+0000","last_became_active":"2024-04-25T13:17:40.818897+0000","last_became_peered":"2024-04-25T13:17:40.818897+0000","last_unstale":"2024-04-25T13:18:33.167636+0000","last_undegraded":"2024-04-25T13:18:33.167636+0000","last_fullsized":"2024-04-25T13:18:33.167636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:15.215991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948406+0000","last_change":"2024-04-25T13:17:40.823789+0000","last_active":"2024-04-25T13:18:32.948406+0000","last_peered":"2024-04-25T13:18:32.948406+0000","last_clean":"2024-04-25T13:18:32.948406+0000","last_became_active":"2024-04-25T13:17:40.823424+0000","last_became_peered":"2024-04-25T13:17:40.823424+0000","last_unstale":"2024-04-25T13:18:32.948406+0000","last_undegraded":"2024-04-25T13:18:32.948406+0000","last_fullsized":"2024-04-25T13:18:32.948406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:50.022286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964529+0000","last_change":"2024-04-25T13:17:40.827085+0000","last_active":"2024-04-25T13:18:32.964529+0000","last_peered":"2024-04-25T13:18:32.964529+0000","last_clean":"2024-04-25T13:18:32.964529+0000","last_became_active":"2024-04-25T13:17:40.826760+0000","last_became_peered":"2024-04-25T13:17:40.826760+0000","last_unstale":"2024-04-25T13:18:32.964529+0000","last_undegraded":"2024-04-25T13:18:32.964529+0000","last_fullsized":"2024-04-25T13:18:32.964529+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:46.662788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964350+0000","last_change":"2024-04-25T13:17:40.824283+0000","last_active":"2024-04-25T13:18:32.964350+0000","last_peered":"2024-04-25T13:18:32.964350+0000","last_clean":"2024-04-25T13:18:32.964350+0000","last_became_active":"2024-04-25T13:17:40.823914+0000","last_became_peered":"2024-04-25T13:17:40.823914+0000","last_unstale":"2024-04-25T13:18:32.964350+0000","last_undegraded":"2024-04-25T13:18:32.964350+0000","last_fullsized":"2024-04-25T13:18:32.964350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:01.158295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.949041+0000","last_change":"2024-04-25T13:17:40.822003+0000","last_active":"2024-04-25T13:18:32.949041+0000","last_peered":"2024-04-25T13:18:32.949041+0000","last_clean":"2024-04-25T13:18:32.949041+0000","last_became_active":"2024-04-25T13:17:40.821453+0000","last_became_peered":"2024-04-25T13:17:40.821453+0000","last_unstale":"2024-04-25T13:18:32.949041+0000","last_undegraded":"2024-04-25T13:18:32.949041+0000","last_fullsized":"2024-04-25T13:18:32.949041+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:34.474332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945558+0000","last_change":"2024-04-25T13:17:40.832449+0000","last_active":"2024-04-25T13:18:32.945558+0000","last_peered":"2024-04-25T13:18:32.945558+0000","last_clean":"2024-04-25T13:18:32.945558+0000","last_became_active":"2024-04-25T13:17:40.832079+0000","last_became_peered":"2024-04-25T13:17:40.832079+0000","last_unstale":"2024-04-25T13:18:32.945558+0000","last_undegraded":"2024-04-25T13:18:32.945558+0000","last_fullsized":"2024-04-25T13:18:32.945558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:44.537129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.951355+0000","last_change":"2024-04-25T13:18:32.951355+0000","last_active":"2024-04-25T13:18:32.951355+0000","last_peered":"2024-04-25T13:18:32.951355+0000","last_clean":"2024-04-25T13:18:32.951355+0000","last_became_active":"2024-04-25T13:18:32.950863+0000","last_became_peered":"2024-04-25T13:18:32.950863+0000","last_unstale":"2024-04-25T13:18:32.951355+0000","last_undegraded":"2024-04-25T13:18:32.951355+0000","last_fullsized":"2024-04-25T13:18:32.951355+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:44.837228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.964929+0000","last_change":"2024-04-25T13:17:40.821024+0000","last_active":"2024-04-25T13:18:32.964929+0000","last_peered":"2024-04-25T13:18:32.964929+0000","last_clean":"2024-04-25T13:18:32.964929+0000","last_became_active":"2024-04-25T13:17:40.820671+0000","last_became_peered":"2024-04-25T13:17:40.820671+0000","last_unstale":"2024-04-25T13:18:32.964929+0000","last_undegraded":"2024-04-25T13:18:32.964929+0000","last_fullsized":"2024-04-25T13:18:32.964929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:57.315116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189806+0000","last_change":"2024-04-25T13:17:40.837131+0000","last_active":"2024-04-25T13:18:33.189806+0000","last_peered":"2024-04-25T13:18:33.189806+0000","last_clean":"2024-04-25T13:18:33.189806+0000","last_became_active":"2024-04-25T13:17:40.836784+0000","last_became_peered":"2024-04-25T13:17:40.836784+0000","last_unstale":"2024-04-25T13:18:33.189806+0000","last_undegraded":"2024-04-25T13:18:33.189806+0000","last_fullsized":"2024-04-25T13:18:33.189806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:41.818001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945994+0000","last_change":"2024-04-25T13:17:40.827149+0000","last_active":"2024-04-25T13:18:32.945994+0000","last_peered":"2024-04-25T13:18:32.945994+0000","last_clean":"2024-04-25T13:18:32.945994+0000","last_became_active":"2024-04-25T13:17:40.826800+0000","last_became_peered":"2024-04-25T13:17:40.826800+0000","last_unstale":"2024-04-25T13:18:32.945994+0000","last_undegraded":"2024-04-25T13:18:32.945994+0000","last_fullsized":"2024-04-25T13:18:32.945994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:46.211785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.189470+0000","last_change":"2024-04-25T13:17:40.825028+0000","last_active":"2024-04-25T13:18:33.189470+0000","last_peered":"2024-04-25T13:18:33.189470+0000","last_clean":"2024-04-25T13:18:33.189470+0000","last_became_active":"2024-04-25T13:17:40.824614+0000","last_became_peered":"2024-04-25T13:17:40.824614+0000","last_unstale":"2024-04-25T13:18:33.189470+0000","last_undegraded":"2024-04-25T13:18:33.189470+0000","last_fullsized":"2024-04-25T13:18:33.189470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:07.629080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:33.169245+0000","last_change":"2024-04-25T13:18:33.169245+0000","last_active":"2024-04-25T13:18:33.169245+0000","last_peered":"2024-04-25T13:18:33.169245+0000","last_clean":"2024-04-25T13:18:33.169245+0000","last_became_active":"2024-04-25T13:18:33.168617+0000","last_became_peered":"2024-04-25T13:18:33.168617+0000","last_unstale":"2024-04-25T13:18:33.169245+0000","last_undegraded":"2024-04-25T13:18:33.169245+0000","last_fullsized":"2024-04-25T13:18:33.169245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:56.190064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.945433+0000","last_change":"2024-04-25T13:17:40.835190+0000","last_active":"2024-04-25T13:18:32.945433+0000","last_peered":"2024-04-25T13:18:32.945433+0000","last_clean":"2024-04-25T13:18:32.945433+0000","last_became_active":"2024-04-25T13:17:40.834847+0000","last_became_peered":"2024-04-25T13:17:40.834847+0000","last_unstale":"2024-04-25T13:18:32.945433+0000","last_undegraded":"2024-04-25T13:18:32.945433+0000","last_fullsized":"2024-04-25T13:18:32.945433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:18.217464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T13:18:32.948768+0000","last_change":"2024-04-25T13:17:40.827578+0000","last_active":"2024-04-25T13:18:32.948768+0000","last_peered":"2024-04-25T13:18:32.948768+0000","last_clean":"2024-04-25T13:18:32.948768+0000","last_became_active":"2024-04-25T13:17:40.827256+0000","last_became_peered":"2024-04-25T13:17:40.827256+0000","last_unstale":"2024-04-25T13:18:32.948768+0000","last_undegraded":"2024-04-25T13:18:32.948768+0000","last_fullsized":"2024-04-25T13:18:32.948768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:17:39.788307+0000","last_clean_scrub_stamp":"2024-04-25T13:17:39.788307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:40.738629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61844,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":153,"num_read_kb":138,"num_write":75,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":185532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285596,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":469141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547232,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1120866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":469723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005100,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":469745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266736,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1060208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430263,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":469258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":530187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":469231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":530221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117062,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":469479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":469163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1585152,"data_stored":1059808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:18:57.006 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T13:18:57.006 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T13:18:57.006 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T13:18:57.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T13:18:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:56 smithi066 ceph-mon[33137]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:56 smithi066 ceph-mon[33137]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:18:57.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:56 smithi136 ceph-mon[33038]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:57.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:56 smithi136 ceph-mon[33038]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:18:57.265 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:58.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.878+0000 7efd96cc7700 1 -- 172.21.15.31:0/526080555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd9006b0f0 msgr2=0x7efd9010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.878+0000 7efd96cc7700 1 --2- 172.21.15.31:0/526080555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd9006b0f0 0x7efd9010f900 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efd8800b0a0 tx=0x7efd8800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.878+0000 7efd96cc7700 1 -- 172.21.15.31:0/526080555 shutdown_connections 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.878+0000 7efd96cc7700 1 --2- 172.21.15.31:0/526080555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd9006b0f0 0x7efd9010f900 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.878+0000 7efd96cc7700 1 --2- 172.21.15.31:0/526080555 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd90108a30 0x7efd90108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.878+0000 7efd96cc7700 1 --2- 172.21.15.31:0/526080555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd9006a750 0x7efd9006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.878+0000 7efd96cc7700 1 -- 172.21.15.31:0/526080555 >> 172.21.15.31:0/526080555 conn(0x7efd900788f0 msgr2=0x7efd90078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.878+0000 7efd96cc7700 1 -- 172.21.15.31:0/526080555 shutdown_connections 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.878+0000 7efd96cc7700 1 -- 172.21.15.31:0/526080555 wait complete. 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.878+0000 7efd96cc7700 1 Processor -- start 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.878+0000 7efd96cc7700 1 -- start start 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd96cc7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd9006a750 0x7efd90199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd96cc7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd9006b0f0 0x7efd90199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd96cc7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd90108a30 0x7efd9019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd96cc7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd90112c20 con 0x7efd90108a30 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd96cc7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd90112aa0 con 0x7efd9006b0f0 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd96cc7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd90112da0 con 0x7efd9006a750 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd95264700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd90108a30 0x7efd9019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd95264700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd90108a30 0x7efd9019d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58160/0 (socket says 172.21.15.31:58160) 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd95264700 1 -- 172.21.15.31:0/3710175224 learned_addr learned my addr 172.21.15.31:0/3710175224 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd94a63700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd9006a750 0x7efd90199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd8ffff700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd9006b0f0 0x7efd90199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd95264700 1 -- 172.21.15.31:0/3710175224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd9006a750 msgr2=0x7efd90199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd95264700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd9006a750 0x7efd90199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd95264700 1 -- 172.21.15.31:0/3710175224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd9006b0f0 msgr2=0x7efd90199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd95264700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd9006b0f0 0x7efd90199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd95264700 1 -- 172.21.15.31:0/3710175224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9019e0f0 con 0x7efd90108a30 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd8ffff700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd9006b0f0 0x7efd90199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:18:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd95264700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd90108a30 0x7efd9019d9b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efd88009fd0 tx=0x7efd88004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.879+0000 7efd8dffb700 1 -- 172.21.15.31:0/3710175224 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88021070 con 0x7efd90108a30 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.880+0000 7efd96cc7700 1 -- 172.21.15.31:0/3710175224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9019e380 con 0x7efd90108a30 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.880+0000 7efd8dffb700 1 -- 172.21.15.31:0/3710175224 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd88007c00 con 0x7efd90108a30 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.880+0000 7efd8dffb700 1 -- 172.21.15.31:0/3710175224 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880052a0 con 0x7efd90108a30 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.880+0000 7efd96cc7700 1 -- 172.21.15.31:0/3710175224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd901a5c60 con 0x7efd90108a30 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.881+0000 7efd96cc7700 1 -- 172.21.15.31:0/3710175224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd90069730 con 0x7efd90108a30 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.882+0000 7efd8dffb700 1 -- 172.21.15.31:0/3710175224 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efd88024070 con 0x7efd90108a30 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.882+0000 7efd8dffb700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efd7c0a3cd0 0x7efd7c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.882+0000 7efd94a63700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efd7c0a3cd0 0x7efd7c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.882+0000 7efd8dffb700 1 -- 172.21.15.31:0/3710175224 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7efd8800f020 con 0x7efd90108a30 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.883+0000 7efd94a63700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efd7c0a3cd0 0x7efd7c0a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efd80007b60 tx=0x7efd800073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:58.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:58.887+0000 7efd8dffb700 1 -- 172.21.15.31:0/3710175224 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd8802b030 con 0x7efd90108a30 2024-04-25T13:18:58.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:58 smithi031 ceph-mon[29457]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:59.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.144+0000 7efd96cc7700 1 -- 172.21.15.31:0/3710175224 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efd9019a440 con 0x7efd90108a30 2024-04-25T13:18:59.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.145+0000 7efd8dffb700 1 -- 172.21.15.31:0/3710175224 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7efd88024e40 con 0x7efd90108a30 2024-04-25T13:18:59.146 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:18:59.146 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T13:18:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 -- 172.21.15.31:0/3710175224 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efd7c0a3cd0 msgr2=0x7efd7c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efd7c0a3cd0 0x7efd7c0a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efd80007b60 tx=0x7efd800073d0 comp rx=0 tx=0).stop 2024-04-25T13:18:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 -- 172.21.15.31:0/3710175224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd90108a30 msgr2=0x7efd9019d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd90108a30 0x7efd9019d9b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efd88009fd0 tx=0x7efd88004dc0 comp rx=0 tx=0).stop 2024-04-25T13:18:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 -- 172.21.15.31:0/3710175224 shutdown_connections 2024-04-25T13:18:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efd7c0a3cd0 0x7efd7c0a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd9006a750 0x7efd90199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd90108a30 0x7efd9019d9b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 --2- 172.21.15.31:0/3710175224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd9006b0f0 0x7efd90199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:59.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 -- 172.21.15.31:0/3710175224 >> 172.21.15.31:0/3710175224 conn(0x7efd900788f0 msgr2=0x7efd900ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:59.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 -- 172.21.15.31:0/3710175224 shutdown_connections 2024-04-25T13:18:59.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.147+0000 7efd96cc7700 1 -- 172.21.15.31:0/3710175224 wait complete. 2024-04-25T13:18:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:58 smithi066 ceph-mon[33137]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:58 smithi136 ceph-mon[33038]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:18:59.804 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T13:18:59.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:18:59.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.918+0000 7f60fe59b640 1 -- 172.21.15.31:0/4108128419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f8105360 msgr2=0x7f60f8107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:59.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.918+0000 7f60fe59b640 1 --2- 172.21.15.31:0/4108128419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f8105360 0x7f60f8107750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009900 tx=0x7f60e802a410 comp rx=0 tx=0).stop 2024-04-25T13:18:59.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.918+0000 7f60fe59b640 1 -- 172.21.15.31:0/4108128419 shutdown_connections 2024-04-25T13:18:59.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.918+0000 7f60fe59b640 1 --2- 172.21.15.31:0/4108128419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f8107c90 0x7f60f810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:59.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.918+0000 7f60fe59b640 1 --2- 172.21.15.31:0/4108128419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f8105360 0x7f60f8107750 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:59.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.918+0000 7f60fe59b640 1 --2- 172.21.15.31:0/4108128419 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60f8102a30 0x7f60f8104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:59.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.918+0000 7f60fe59b640 1 -- 172.21.15.31:0/4108128419 >> 172.21.15.31:0/4108128419 conn(0x7f60f80fc440 msgr2=0x7f60f80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:59.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.918+0000 7f60fe59b640 1 -- 172.21.15.31:0/4108128419 shutdown_connections 2024-04-25T13:18:59.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.918+0000 7f60fe59b640 1 -- 172.21.15.31:0/4108128419 wait complete. 2024-04-25T13:18:59.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.919+0000 7f60fe59b640 1 Processor -- start 2024-04-25T13:18:59.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.919+0000 7f60fe59b640 1 -- start start 2024-04-25T13:18:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.919+0000 7f60fe59b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f8102a30 0x7f60f8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.919+0000 7f60fe59b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60f8105360 0x7f60f8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.919+0000 7f60fe59b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f8107c90 0x7f60f819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.919+0000 7f60fe59b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60f810db40 con 0x7f60f8102a30 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.919+0000 7f60fe59b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f60f810d9c0 con 0x7f60f8105360 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.919+0000 7f60fe59b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60f810dcc0 con 0x7f60f8107c90 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.919+0000 7f60f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f8102a30 0x7f60f8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.919+0000 7f60f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f8102a30 0x7f60f8198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58176/0 (socket says 172.21.15.31:58176) 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.919+0000 7f60f7fff640 1 -- 172.21.15.31:0/2889243650 learned_addr learned my addr 172.21.15.31:0/2889243650 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60fcb38640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f8107c90 0x7f60f819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60f77fe640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60f8105360 0x7f60f8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60f7fff640 1 -- 172.21.15.31:0/2889243650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f8107c90 msgr2=0x7f60f819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60f7fff640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f8107c90 0x7f60f819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60f7fff640 1 -- 172.21.15.31:0/2889243650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60f8105360 msgr2=0x7f60f8198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60f7fff640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60f8105360 0x7f60f8198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60f7fff640 1 -- 172.21.15.31:0/2889243650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f819fe20 con 0x7f60f8102a30 2024-04-25T13:18:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60f7fff640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f8102a30 0x7f60f8198200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00c890 tx=0x7f60ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60f57fa640 1 -- 172.21.15.31:0/2889243650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec004220 con 0x7f60f8102a30 2024-04-25T13:18:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60f57fa640 1 -- 172.21.15.31:0/2889243650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60ec0043c0 con 0x7f60f8102a30 2024-04-25T13:18:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60fe59b640 1 -- 172.21.15.31:0/2889243650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f81a0110 con 0x7f60f8102a30 2024-04-25T13:18:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60f57fa640 1 -- 172.21.15.31:0/2889243650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec004a00 con 0x7f60f8102a30 2024-04-25T13:18:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.920+0000 7f60fe59b640 1 -- 172.21.15.31:0/2889243650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f81a0630 con 0x7f60f8102a30 2024-04-25T13:18:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.921+0000 7f60fe59b640 1 -- 172.21.15.31:0/2889243650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f80fdf30 con 0x7f60f8102a30 2024-04-25T13:18:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.923+0000 7f60f57fa640 1 -- 172.21.15.31:0/2889243650 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f60ec0079b0 con 0x7f60f8102a30 2024-04-25T13:18:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.923+0000 7f60f57fa640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60e00a3c40 0x7f60e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.923+0000 7f60f77fe640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60e00a3c40 0x7f60e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.923+0000 7f60f57fa640 1 -- 172.21.15.31:0/2889243650 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f60ec0d02f0 con 0x7f60f8102a30 2024-04-25T13:18:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.924+0000 7f60f77fe640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60e00a3c40 0x7f60e00a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006010 tx=0x7f60e80314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:18:59.928+0000 7f60f57fa640 1 -- 172.21.15.31:0/2889243650 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60ec014030 con 0x7f60f8102a30 2024-04-25T13:19:00.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.117+0000 7f60fe59b640 1 -- 172.21.15.31:0/2889243650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60f8062b50 con 0x7f60f8102a30 2024-04-25T13:19:00.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.118+0000 7f60f57fa640 1 -- 172.21.15.31:0/2889243650 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22650 (secure 0 0 0) 0x7f60ec09cb50 con 0x7f60f8102a30 2024-04-25T13:19:00.119 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:00.120 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:18:32.938284+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.2e","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:19:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 -- 172.21.15.31:0/2889243650 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60e00a3c40 msgr2=0x7f60e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60e00a3c40 0x7f60e00a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006010 tx=0x7f60e80314a0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 -- 172.21.15.31:0/2889243650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f8102a30 msgr2=0x7f60f8198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f8102a30 0x7f60f8198200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00c890 tx=0x7f60ec00cd50 comp rx=0 tx=0).stop 2024-04-25T13:19:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 -- 172.21.15.31:0/2889243650 shutdown_connections 2024-04-25T13:19:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f60e00a3c40 0x7f60e00a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f8107c90 0x7f60f819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60f8105360 0x7f60f8198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 --2- 172.21.15.31:0/2889243650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f8102a30 0x7f60f8198200 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 -- 172.21.15.31:0/2889243650 >> 172.21.15.31:0/2889243650 conn(0x7f60f80fc440 msgr2=0x7f60f81032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 -- 172.21.15.31:0/2889243650 shutdown_connections 2024-04-25T13:19:00.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.121+0000 7f60fe59b640 1 -- 172.21.15.31:0/2889243650 wait complete. 2024-04-25T13:19:00.131 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:18:59 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3710175224' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:19:00.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:13:01.793580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:17:37.994125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:17:39.116456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:19:00.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:19:00.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:18:59 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3710175224' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:19:00.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:18:59 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3710175224' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:19:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.237+0000 7f80a906b640 1 -- 172.21.15.31:0/1413627919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4105340 msgr2=0x7f80a4107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.237+0000 7f80a906b640 1 --2- 172.21.15.31:0/1413627919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4105340 0x7f80a4107730 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8098009900 tx=0x7f809802a410 comp rx=0 tx=0).stop 2024-04-25T13:19:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.237+0000 7f80a906b640 1 -- 172.21.15.31:0/1413627919 shutdown_connections 2024-04-25T13:19:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.237+0000 7f80a906b640 1 --2- 172.21.15.31:0/1413627919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a4107c70 0x7f80a410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.237+0000 7f80a906b640 1 --2- 172.21.15.31:0/1413627919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4105340 0x7f80a4107730 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.237+0000 7f80a906b640 1 --2- 172.21.15.31:0/1413627919 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a4102a10 0x7f80a4104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.237+0000 7f80a906b640 1 -- 172.21.15.31:0/1413627919 >> 172.21.15.31:0/1413627919 conn(0x7f80a40fc400 msgr2=0x7f80a40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.237+0000 7f80a906b640 1 -- 172.21.15.31:0/1413627919 shutdown_connections 2024-04-25T13:19:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.237+0000 7f80a906b640 1 -- 172.21.15.31:0/1413627919 wait complete. 2024-04-25T13:19:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.238+0000 7f80a906b640 1 Processor -- start 2024-04-25T13:19:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.238+0000 7f80a906b640 1 -- start start 2024-04-25T13:19:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.238+0000 7f80a906b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a4102a10 0x7f80a4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.238+0000 7f80a906b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4105340 0x7f80a4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.238+0000 7f80a906b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a4107c70 0x7f80a419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.238+0000 7f80a906b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f80a410dab0 con 0x7f80a4105340 2024-04-25T13:19:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.238+0000 7f80a906b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f80a410d930 con 0x7f80a4107c70 2024-04-25T13:19:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.238+0000 7f80a906b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80a410dc30 con 0x7f80a4102a10 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.238+0000 7f80a259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4105340 0x7f80a4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.238+0000 7f80a259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4105340 0x7f80a4198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58194/0 (socket says 172.21.15.31:58194) 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.238+0000 7f80a259c640 1 -- 172.21.15.31:0/3481961826 learned_addr learned my addr 172.21.15.31:0/3481961826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f80a2d9d640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a4102a10 0x7f80a4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f80a359e640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a4107c70 0x7f80a419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f80a259c640 1 -- 172.21.15.31:0/3481961826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a4102a10 msgr2=0x7f80a4198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f80a259c640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a4102a10 0x7f80a4198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f80a259c640 1 -- 172.21.15.31:0/3481961826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a4107c70 msgr2=0x7f80a419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f80a259c640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a4107c70 0x7f80a419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f80a259c640 1 -- 172.21.15.31:0/3481961826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a419fe50 con 0x7f80a4105340 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f80a359e640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a4107c70 0x7f80a419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f80a259c640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4105340 0x7f80a4198770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8098006010 tx=0x7f80980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f808bfff640 1 -- 172.21.15.31:0/3481961826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809803f030 con 0x7f80a4105340 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f808bfff640 1 -- 172.21.15.31:0/3481961826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8098003cb0 con 0x7f80a4105340 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f80a906b640 1 -- 172.21.15.31:0/3481961826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a41a00e0 con 0x7f80a4105340 2024-04-25T13:19:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f808bfff640 1 -- 172.21.15.31:0/3481961826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098035070 con 0x7f80a4105340 2024-04-25T13:19:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.239+0000 7f80a906b640 1 -- 172.21.15.31:0/3481961826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a41a05d0 con 0x7f80a4105340 2024-04-25T13:19:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.240+0000 7f80a906b640 1 -- 172.21.15.31:0/3481961826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a40fdef0 con 0x7f80a4105340 2024-04-25T13:19:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.241+0000 7f808bfff640 1 -- 172.21.15.31:0/3481961826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8098042030 con 0x7f80a4105340 2024-04-25T13:19:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.242+0000 7f808bfff640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f808c0a3c40 0x7f808c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.242+0000 7f80a2d9d640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f808c0a3c40 0x7f808c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.242+0000 7f808bfff640 1 -- 172.21.15.31:0/3481961826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f80980f0c50 con 0x7f80a4105340 2024-04-25T13:19:00.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.242+0000 7f80a2d9d640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f808c0a3c40 0x7f808c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8094005fd0 tx=0x7f8094009760 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:00.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.246+0000 7f808bfff640 1 -- 172.21.15.31:0/3481961826 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80980bd450 con 0x7f80a4105340 2024-04-25T13:19:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.431+0000 7f80a906b640 1 -- 172.21.15.31:0/3481961826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f80a402d810 con 0x7f80a4105340 2024-04-25T13:19:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.432+0000 7f808bfff640 1 -- 172.21.15.31:0/3481961826 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f80980bcd50 con 0x7f80a4105340 2024-04-25T13:19:00.433 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T13:19:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 -- 172.21.15.31:0/3481961826 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f808c0a3c40 msgr2=0x7f808c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f808c0a3c40 0x7f808c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8094005fd0 tx=0x7f8094009760 comp rx=0 tx=0).stop 2024-04-25T13:19:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 -- 172.21.15.31:0/3481961826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4105340 msgr2=0x7f80a4198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4105340 0x7f80a4198770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8098006010 tx=0x7f80980042f0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 -- 172.21.15.31:0/3481961826 shutdown_connections 2024-04-25T13:19:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f808c0a3c40 0x7f808c0a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80a4107c70 0x7f80a419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80a4105340 0x7f80a4198770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 --2- 172.21.15.31:0/3481961826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a4102a10 0x7f80a4198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 -- 172.21.15.31:0/3481961826 >> 172.21.15.31:0/3481961826 conn(0x7f80a40fc400 msgr2=0x7f80a41048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 -- 172.21.15.31:0/3481961826 shutdown_connections 2024-04-25T13:19:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.434+0000 7f80a906b640 1 -- 172.21.15.31:0/3481961826 wait complete. 2024-04-25T13:19:00.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:19:00.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.549+0000 7f81b10e1640 1 -- 172.21.15.31:0/177391516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81ac101fb0 msgr2=0x7f81ac1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.549+0000 7f81b10e1640 1 --2- 172.21.15.31:0/177391516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81ac101fb0 0x7f81ac1023d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f819c009900 tx=0x7f819c02a410 comp rx=0 tx=0).stop 2024-04-25T13:19:00.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.549+0000 7f81b10e1640 1 -- 172.21.15.31:0/177391516 shutdown_connections 2024-04-25T13:19:00.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.549+0000 7f81b10e1640 1 --2- 172.21.15.31:0/177391516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81ac103b60 0x7f81ac10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.549+0000 7f81b10e1640 1 --2- 172.21.15.31:0/177391516 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81ac1031a0 0x7f81ac103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.549+0000 7f81b10e1640 1 --2- 172.21.15.31:0/177391516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81ac101fb0 0x7f81ac1023d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.549+0000 7f81b10e1640 1 -- 172.21.15.31:0/177391516 >> 172.21.15.31:0/177391516 conn(0x7f81ac0fd540 msgr2=0x7f81ac0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:00.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.549+0000 7f81b10e1640 1 -- 172.21.15.31:0/177391516 shutdown_connections 2024-04-25T13:19:00.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.549+0000 7f81b10e1640 1 -- 172.21.15.31:0/177391516 wait complete. 2024-04-25T13:19:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.550+0000 7f81b10e1640 1 Processor -- start 2024-04-25T13:19:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.550+0000 7f81b10e1640 1 -- start start 2024-04-25T13:19:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.550+0000 7f81b10e1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81ac101fb0 0x7f81ac198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.550+0000 7f81b10e1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81ac1031a0 0x7f81ac198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.550+0000 7f81b10e1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81ac103b60 0x7f81ac19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.550+0000 7f81b10e1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10db30 con 0x7f81ac103b60 2024-04-25T13:19:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.550+0000 7f81b10e1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10d9b0 con 0x7f81ac1031a0 2024-04-25T13:19:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.550+0000 7f81b10e1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10dcb0 con 0x7f81ac101fb0 2024-04-25T13:19:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.550+0000 7f81ab59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81ac103b60 0x7f81ac19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81ab59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81ac103b60 0x7f81ac19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58212/0 (socket says 172.21.15.31:58212) 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81ab59e640 1 -- 172.21.15.31:0/267944722 learned_addr learned my addr 172.21.15.31:0/267944722 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81aa59c640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81ac1031a0 0x7f81ac198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81aad9d640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81ac101fb0 0x7f81ac198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81ab59e640 1 -- 172.21.15.31:0/267944722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81ac101fb0 msgr2=0x7f81ac198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81ab59e640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81ac101fb0 0x7f81ac198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81ab59e640 1 -- 172.21.15.31:0/267944722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81ac1031a0 msgr2=0x7f81ac198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81ab59e640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81ac1031a0 0x7f81ac198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81ab59e640 1 -- 172.21.15.31:0/267944722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac19fe50 con 0x7f81ac103b60 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81ab59e640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81ac103b60 0x7f81ac19f7e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f81a400eaa0 tx=0x7f81a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f8193fff640 1 -- 172.21.15.31:0/267944722 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a400cdb0 con 0x7f81ac103b60 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f8193fff640 1 -- 172.21.15.31:0/267944722 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a4005020 con 0x7f81ac103b60 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81b10e1640 1 -- 172.21.15.31:0/267944722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ac1a0140 con 0x7f81ac103b60 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f8193fff640 1 -- 172.21.15.31:0/267944722 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a40116d0 con 0x7f81ac103b60 2024-04-25T13:19:00.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.551+0000 7f81b10e1640 1 -- 172.21.15.31:0/267944722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ac1a06c0 con 0x7f81ac103b60 2024-04-25T13:19:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.552+0000 7f81b10e1640 1 -- 172.21.15.31:0/267944722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ac1093a0 con 0x7f81ac103b60 2024-04-25T13:19:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.553+0000 7f8193fff640 1 -- 172.21.15.31:0/267944722 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f81a40051d0 con 0x7f81ac103b60 2024-04-25T13:19:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.554+0000 7f8193fff640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f81940a3c40 0x7f81940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.554+0000 7f81aad9d640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f81940a3c40 0x7f81940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.554+0000 7f8193fff640 1 -- 172.21.15.31:0/267944722 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f81a40d08c0 con 0x7f81ac103b60 2024-04-25T13:19:00.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.555+0000 7f81aad9d640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f81940a3c40 0x7f81940a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f819c02a930 tx=0x7f819c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:00.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.559+0000 7f8193fff640 1 -- 172.21.15.31:0/267944722 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81a4016030 con 0x7f81ac103b60 2024-04-25T13:19:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.746+0000 7f81b10e1640 1 -- 172.21.15.31:0/267944722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f81ac062b50 con 0x7f81ac103b60 2024-04-25T13:19:00.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.747+0000 7f8193fff640 1 -- 172.21.15.31:0/267944722 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f81a409d0c0 con 0x7f81ac103b60 2024-04-25T13:19:00.748 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T13:19:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 -- 172.21.15.31:0/267944722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f81940a3c40 msgr2=0x7f81940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f81940a3c40 0x7f81940a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f819c02a930 tx=0x7f819c000bc0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 -- 172.21.15.31:0/267944722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81ac103b60 msgr2=0x7f81ac19f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81ac103b60 0x7f81ac19f7e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f81a400eaa0 tx=0x7f81a400ef60 comp rx=0 tx=0).stop 2024-04-25T13:19:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 -- 172.21.15.31:0/267944722 shutdown_connections 2024-04-25T13:19:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f81940a3c40 0x7f81940a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81ac103b60 0x7f81ac19f7e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f81ac1031a0 0x7f81ac198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 --2- 172.21.15.31:0/267944722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81ac101fb0 0x7f81ac198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 -- 172.21.15.31:0/267944722 >> 172.21.15.31:0/267944722 conn(0x7f81ac0fd540 msgr2=0x7f81ac106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 -- 172.21.15.31:0/267944722 shutdown_connections 2024-04-25T13:19:00.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.749+0000 7f81b10e1640 1 -- 172.21.15.31:0/267944722 wait complete. 2024-04-25T13:19:00.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:19:00.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.867+0000 7f3892cbb640 1 -- 172.21.15.31:0/492215270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c102040 msgr2=0x7f388c1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.867+0000 7f3892cbb640 1 --2- 172.21.15.31:0/492215270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c102040 0x7f388c1024c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3878009900 tx=0x7f387802a410 comp rx=0 tx=0).stop 2024-04-25T13:19:00.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.867+0000 7f3892cbb640 1 -- 172.21.15.31:0/492215270 shutdown_connections 2024-04-25T13:19:00.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.867+0000 7f3892cbb640 1 --2- 172.21.15.31:0/492215270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f388c102a00 0x7f388c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.867+0000 7f3892cbb640 1 --2- 172.21.15.31:0/492215270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c102040 0x7f388c1024c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.867+0000 7f3892cbb640 1 --2- 172.21.15.31:0/492215270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f388c100e60 0x7f388c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.867+0000 7f3892cbb640 1 -- 172.21.15.31:0/492215270 >> 172.21.15.31:0/492215270 conn(0x7f388c0fc420 msgr2=0x7f388c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.867+0000 7f3892cbb640 1 -- 172.21.15.31:0/492215270 shutdown_connections 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.867+0000 7f3892cbb640 1 -- 172.21.15.31:0/492215270 wait complete. 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3892cbb640 1 Processor -- start 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3892cbb640 1 -- start start 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3892cbb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f388c100e60 0x7f388c06b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3892cbb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f388c102040 0x7f388c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3892cbb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c102a00 0x7f388c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3892cbb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f388c10c9f0 con 0x7f388c102a00 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3892cbb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f388c10c870 con 0x7f388c102040 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3892cbb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f388c10cb70 con 0x7f388c100e60 2024-04-25T13:19:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3891258640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c102a00 0x7f388c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3891258640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c102a00 0x7f388c19f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58234/0 (socket says 172.21.15.31:58234) 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3891258640 1 -- 172.21.15.31:0/309309255 learned_addr learned my addr 172.21.15.31:0/309309255 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3890a57640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f388c100e60 0x7f388c06b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f388bfff640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f388c102040 0x7f388c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3891258640 1 -- 172.21.15.31:0/309309255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f388c100e60 msgr2=0x7f388c06b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3891258640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f388c100e60 0x7f388c06b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3891258640 1 -- 172.21.15.31:0/309309255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f388c102040 msgr2=0x7f388c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3891258640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f388c102040 0x7f388c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3891258640 1 -- 172.21.15.31:0/309309255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c19fda0 con 0x7f388c102a00 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.868+0000 7f3891258640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c102a00 0x7f388c19f4e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f388400cce0 tx=0x7f388400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.869+0000 7f3889ffb640 1 -- 172.21.15.31:0/309309255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388400f450 con 0x7f388c102a00 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.869+0000 7f3889ffb640 1 -- 172.21.15.31:0/309309255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388400faa0 con 0x7f388c102a00 2024-04-25T13:19:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.869+0000 7f3892cbb640 1 -- 172.21.15.31:0/309309255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c1a0090 con 0x7f388c102a00 2024-04-25T13:19:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.869+0000 7f3889ffb640 1 -- 172.21.15.31:0/309309255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884018b70 con 0x7f388c102a00 2024-04-25T13:19:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.869+0000 7f3892cbb640 1 -- 172.21.15.31:0/309309255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c1a05e0 con 0x7f388c102a00 2024-04-25T13:19:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.870+0000 7f3892cbb640 1 -- 172.21.15.31:0/309309255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388c108280 con 0x7f388c102a00 2024-04-25T13:19:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.871+0000 7f3889ffb640 1 -- 172.21.15.31:0/309309255 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3884018d10 con 0x7f388c102a00 2024-04-25T13:19:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.871+0000 7f3889ffb640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f387c0a3cc0 0x7f387c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.871+0000 7f3890a57640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f387c0a3cc0 0x7f387c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.871+0000 7f3889ffb640 1 -- 172.21.15.31:0/309309255 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f38840d0980 con 0x7f388c102a00 2024-04-25T13:19:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.872+0000 7f3890a57640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f387c0a3cc0 0x7f387c0a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f388c101bb0 tx=0x7f388000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:00.876+0000 7f3889ffb640 1 -- 172.21.15.31:0/309309255 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3884014030 con 0x7f388c102a00 2024-04-25T13:19:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.064+0000 7f3892cbb640 1 -- 172.21.15.31:0/309309255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f388c02d7a0 con 0x7f388c102a00 2024-04-25T13:19:01.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.064+0000 7f3889ffb640 1 -- 172.21.15.31:0/309309255 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f388409d1e0 con 0x7f388c102a00 2024-04-25T13:19:01.065 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T13:19:01.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 -- 172.21.15.31:0/309309255 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f387c0a3cc0 msgr2=0x7f387c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f387c0a3cc0 0x7f387c0a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f388c101bb0 tx=0x7f388000a2d0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 -- 172.21.15.31:0/309309255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c102a00 msgr2=0x7f388c19f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c102a00 0x7f388c19f4e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f388400cce0 tx=0x7f388400a620 comp rx=0 tx=0).stop 2024-04-25T13:19:01.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 -- 172.21.15.31:0/309309255 shutdown_connections 2024-04-25T13:19:01.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f387c0a3cc0 0x7f387c0a6180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f388c102a00 0x7f388c19f4e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f388c102040 0x7f388c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 --2- 172.21.15.31:0/309309255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f388c100e60 0x7f388c06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 -- 172.21.15.31:0/309309255 >> 172.21.15.31:0/309309255 conn(0x7f388c0fc420 msgr2=0x7f388c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:01.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 -- 172.21.15.31:0/309309255 shutdown_connections 2024-04-25T13:19:01.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.066+0000 7f3892cbb640 1 -- 172.21.15.31:0/309309255 wait complete. 2024-04-25T13:19:01.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:00 smithi031 ceph-mon[29457]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:19:01.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:00 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2889243650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:01.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:00 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3481961826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:01.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:00 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/267944722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:01.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:19:01.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:00 smithi066 ceph-mon[33137]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:19:01.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:00 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2889243650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:01.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:00 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3481961826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:01.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:00 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/267944722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:01.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.179+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/3233879425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64100e60 msgr2=0x7f3f64101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.179+0000 7f3f6b0b9640 1 --2- 172.21.15.31:0/3233879425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64100e60 0x7f3f64101280 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009900 tx=0x7f3f5802a410 comp rx=0 tx=0).stop 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.179+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/3233879425 shutdown_connections 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.179+0000 7f3f6b0b9640 1 --2- 172.21.15.31:0/3233879425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f64102a00 0x7f3f64109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.179+0000 7f3f6b0b9640 1 --2- 172.21.15.31:0/3233879425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f64102040 0x7f3f641024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.179+0000 7f3f6b0b9640 1 --2- 172.21.15.31:0/3233879425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64100e60 0x7f3f64101280 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.179+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/3233879425 >> 172.21.15.31:0/3233879425 conn(0x7f3f640fc440 msgr2=0x7f3f640fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.179+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/3233879425 shutdown_connections 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.179+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/3233879425 wait complete. 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.179+0000 7f3f6b0b9640 1 Processor -- start 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.179+0000 7f3f6b0b9640 1 -- start start 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f6b0b9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f64100e60 0x7f3f6419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f6b0b9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f64102040 0x7f3f6419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f6b0b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64102a00 0x7f3f641a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f6b0b9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6410c9d0 con 0x7f3f64102a00 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f6b0b9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6410c850 con 0x7f3f64102040 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f6b0b9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6410cb50 con 0x7f3f64100e60 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f69656640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64102a00 0x7f3f641a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f69656640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64102a00 0x7f3f641a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58246/0 (socket says 172.21.15.31:58246) 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f69656640 1 -- 172.21.15.31:0/1144466970 learned_addr learned my addr 172.21.15.31:0/1144466970 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f63fff640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f64102040 0x7f3f6419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f68e55640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f64100e60 0x7f3f6419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f69656640 1 -- 172.21.15.31:0/1144466970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f64100e60 msgr2=0x7f3f6419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f69656640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f64100e60 0x7f3f6419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f69656640 1 -- 172.21.15.31:0/1144466970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f64102040 msgr2=0x7f3f6419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f69656640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f64102040 0x7f3f6419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f69656640 1 -- 172.21.15.31:0/1144466970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f641a1fc0 con 0x7f3f64102a00 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f63fff640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f64102040 0x7f3f6419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:01.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f69656640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64102a00 0x7f3f641a1950 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00cce0 tx=0x7f3f5c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f61ffb640 1 -- 172.21.15.31:0/1144466970 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00f450 con 0x7f3f64102a00 2024-04-25T13:19:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/1144466970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f641a22b0 con 0x7f3f64102a00 2024-04-25T13:19:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f61ffb640 1 -- 172.21.15.31:0/1144466970 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f5c00faa0 con 0x7f3f64102a00 2024-04-25T13:19:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.180+0000 7f3f61ffb640 1 -- 172.21.15.31:0/1144466970 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c018b70 con 0x7f3f64102a00 2024-04-25T13:19:01.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.181+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/1144466970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f641a27b0 con 0x7f3f64102a00 2024-04-25T13:19:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.181+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/1144466970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f641a2a70 con 0x7f3f64102a00 2024-04-25T13:19:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.182+0000 7f3f61ffb640 1 -- 172.21.15.31:0/1144466970 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3f5c018d10 con 0x7f3f64102a00 2024-04-25T13:19:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.183+0000 7f3f61ffb640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3f540a3c40 0x7f3f540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.183+0000 7f3f68e55640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3f540a3c40 0x7f3f540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.183+0000 7f3f61ffb640 1 -- 172.21.15.31:0/1144466970 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f3f5c0d0aa0 con 0x7f3f64102a00 2024-04-25T13:19:01.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.184+0000 7f3f68e55640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3f540a3c40 0x7f3f540a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3f58000c00 tx=0x7f3f580314c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.188+0000 7f3f61ffb640 1 -- 172.21.15.31:0/1144466970 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f5c014030 con 0x7f3f64102a00 2024-04-25T13:19:01.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:00 smithi136 ceph-mon[33038]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:19:01.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:00 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2889243650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:01.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:00 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3481961826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:01.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:00 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/267944722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:01.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.414+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/1144466970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f641a2c40 con 0x7f3f64102a00 2024-04-25T13:19:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.415+0000 7f3f61ffb640 1 -- 172.21.15.31:0/1144466970 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11798 (secure 0 0 0) 0x7f3f5c09d300 con 0x7f3f64102a00 2024-04-25T13:19:01.416 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:01.416 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2599371925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2599371925},{"type":"v1","addr":"172.21.15.31:6839","nonce":2599371925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/164386980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":164386980},{"type":"v1","addr":"172.21.15.31:6841","nonce":164386980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:17:40.162672+0000","modified":"2024-04-25T13:17:44.194940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2682191586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2682191586},{"type":"v1","addr":"172.21.15.66:6839","nonce":2682191586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24511]},"id":1}]} 2024-04-25T13:19:01.416 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-04-25T13:19:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/1144466970 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3f540a3c40 msgr2=0x7f3f540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3f540a3c40 0x7f3f540a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3f58000c00 tx=0x7f3f580314c0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/1144466970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64102a00 msgr2=0x7f3f641a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64102a00 0x7f3f641a1950 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00cce0 tx=0x7f3f5c00a620 comp rx=0 tx=0).stop 2024-04-25T13:19:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/1144466970 shutdown_connections 2024-04-25T13:19:01.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3f540a3c40 0x7f3f540a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f64102a00 0x7f3f641a1950 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f64102040 0x7f3f6419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 --2- 172.21.15.31:0/1144466970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f64100e60 0x7f3f6419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/1144466970 >> 172.21.15.31:0/1144466970 conn(0x7f3f640fc440 msgr2=0x7f3f640fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:01.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/1144466970 shutdown_connections 2024-04-25T13:19:01.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.417+0000 7f3f6b0b9640 1 -- 172.21.15.31:0/1144466970 wait complete. 2024-04-25T13:19:01.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:19:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.530+0000 7f76da260640 1 -- 172.21.15.31:0/983053189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76d4101fa0 msgr2=0x7f76d41023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.530+0000 7f76da260640 1 --2- 172.21.15.31:0/983053189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76d4101fa0 0x7f76d41023c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009900 tx=0x7f76c402a410 comp rx=0 tx=0).stop 2024-04-25T13:19:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.530+0000 7f76da260640 1 -- 172.21.15.31:0/983053189 shutdown_connections 2024-04-25T13:19:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.530+0000 7f76da260640 1 --2- 172.21.15.31:0/983053189 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d4103b50 0x7f76d410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.530+0000 7f76da260640 1 --2- 172.21.15.31:0/983053189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d4103190 0x7f76d4103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.530+0000 7f76da260640 1 --2- 172.21.15.31:0/983053189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76d4101fa0 0x7f76d41023c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.530+0000 7f76da260640 1 -- 172.21.15.31:0/983053189 >> 172.21.15.31:0/983053189 conn(0x7f76d40fd530 msgr2=0x7f76d40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.530+0000 7f76da260640 1 -- 172.21.15.31:0/983053189 shutdown_connections 2024-04-25T13:19:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.530+0000 7f76da260640 1 -- 172.21.15.31:0/983053189 wait complete. 2024-04-25T13:19:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.531+0000 7f76da260640 1 Processor -- start 2024-04-25T13:19:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.531+0000 7f76da260640 1 -- start start 2024-04-25T13:19:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.531+0000 7f76da260640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d4101fa0 0x7f76d4198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.531+0000 7f76da260640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d4103190 0x7f76d4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.531+0000 7f76da260640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76d4103b50 0x7f76d419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.531+0000 7f76da260640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f76d410dae0 con 0x7f76d4103b50 2024-04-25T13:19:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.531+0000 7f76da260640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76d410d960 con 0x7f76d4103190 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.531+0000 7f76da260640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76d410dc60 con 0x7f76d4101fa0 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.531+0000 7f76d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76d4103b50 0x7f76d419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.531+0000 7f76d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76d4103b50 0x7f76d419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58266/0 (socket says 172.21.15.31:58266) 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d3fff640 1 -- 172.21.15.31:0/3017387810 learned_addr learned my addr 172.21.15.31:0/3017387810 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d37fe640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d4101fa0 0x7f76d4198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d2ffd640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d4103190 0x7f76d4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d3fff640 1 -- 172.21.15.31:0/3017387810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d4101fa0 msgr2=0x7f76d4198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d3fff640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d4101fa0 0x7f76d4198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d3fff640 1 -- 172.21.15.31:0/3017387810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d4103190 msgr2=0x7f76d4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d3fff640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d4103190 0x7f76d4198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d3fff640 1 -- 172.21.15.31:0/3017387810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d419fe60 con 0x7f76d4103b50 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d3fff640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76d4103b50 0x7f76d419f7f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00cad0 tx=0x7f76cc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d0ff9640 1 -- 172.21.15.31:0/3017387810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00ae20 con 0x7f76d4103b50 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d0ff9640 1 -- 172.21.15.31:0/3017387810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76cc005020 con 0x7f76d4103b50 2024-04-25T13:19:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76da260640 1 -- 172.21.15.31:0/3017387810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d41a0150 con 0x7f76d4103b50 2024-04-25T13:19:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76d0ff9640 1 -- 172.21.15.31:0/3017387810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00f6b0 con 0x7f76d4103b50 2024-04-25T13:19:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.532+0000 7f76da260640 1 -- 172.21.15.31:0/3017387810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41a0650 con 0x7f76d4103b50 2024-04-25T13:19:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.533+0000 7f76da260640 1 -- 172.21.15.31:0/3017387810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d4109390 con 0x7f76d4103b50 2024-04-25T13:19:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.534+0000 7f76d0ff9640 1 -- 172.21.15.31:0/3017387810 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f76cc0051d0 con 0x7f76d4103b50 2024-04-25T13:19:01.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.535+0000 7f76d0ff9640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f76bc0a3c40 0x7f76bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.535+0000 7f76d37fe640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f76bc0a3c40 0x7f76bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.535+0000 7f76d0ff9640 1 -- 172.21.15.31:0/3017387810 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f76cc0d0670 con 0x7f76d4103b50 2024-04-25T13:19:01.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.535+0000 7f76d37fe640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f76bc0a3c40 0x7f76bc0a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009230 tx=0x7f76c40314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:01.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.539+0000 7f76d0ff9640 1 -- 172.21.15.31:0/3017387810 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76cc014030 con 0x7f76d4103b50 2024-04-25T13:19:01.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.725+0000 7f76da260640 1 -- 172.21.15.31:0/3017387810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76d4062b50 con 0x7f76d4103b50 2024-04-25T13:19:01.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.727+0000 7f76d0ff9640 1 -- 172.21.15.31:0/3017387810 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22650 (secure 0 0 0) 0x7f76cc09ced0 con 0x7f76d4103b50 2024-04-25T13:19:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:01.728 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:18:32.938284+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.2e","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:19:01.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 -- 172.21.15.31:0/3017387810 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f76bc0a3c40 msgr2=0x7f76bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f76bc0a3c40 0x7f76bc0a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009230 tx=0x7f76c40314a0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 -- 172.21.15.31:0/3017387810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76d4103b50 msgr2=0x7f76d419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76d4103b50 0x7f76d419f7f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00cad0 tx=0x7f76cc00cf90 comp rx=0 tx=0).stop 2024-04-25T13:19:01.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 -- 172.21.15.31:0/3017387810 shutdown_connections 2024-04-25T13:19:01.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f76bc0a3c40 0x7f76bc0a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76d4103b50 0x7f76d419f7f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76d4103190 0x7f76d4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 --2- 172.21.15.31:0/3017387810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d4101fa0 0x7f76d4198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 -- 172.21.15.31:0/3017387810 >> 172.21.15.31:0/3017387810 conn(0x7f76d40fd530 msgr2=0x7f76d4106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 -- 172.21.15.31:0/3017387810 shutdown_connections 2024-04-25T13:19:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.729+0000 7f76da260640 1 -- 172.21.15.31:0/3017387810 wait complete. 2024-04-25T13:19:01.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:13:01.793580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:17:37.994125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:17:39.116456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:19:01.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:19:01.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.867+0000 7f4337a72640 1 -- 172.21.15.31:0/1898893950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4330075f10 msgr2=0x7f4330074360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.867+0000 7f4337a72640 1 --2- 172.21.15.31:0/1898893950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4330075f10 0x7f4330074360 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f432c009900 tx=0x7f432c02a480 comp rx=0 tx=0).stop 2024-04-25T13:19:01.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.867+0000 7f4337a72640 1 -- 172.21.15.31:0/1898893950 shutdown_connections 2024-04-25T13:19:01.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.867+0000 7f4337a72640 1 --2- 172.21.15.31:0/1898893950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4330109e50 0x7f433010c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.867+0000 7f4337a72640 1 --2- 172.21.15.31:0/1898893950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4330074930 0x7f4330074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.867+0000 7f4337a72640 1 --2- 172.21.15.31:0/1898893950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4330075f10 0x7f4330074360 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.867+0000 7f4337a72640 1 -- 172.21.15.31:0/1898893950 >> 172.21.15.31:0/1898893950 conn(0x7f43300fd530 msgr2=0x7f43300ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.867+0000 7f4337a72640 1 -- 172.21.15.31:0/1898893950 shutdown_connections 2024-04-25T13:19:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.867+0000 7f4337a72640 1 -- 172.21.15.31:0/1898893950 wait complete. 2024-04-25T13:19:01.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f4337a72640 1 Processor -- start 2024-04-25T13:19:01.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f4337a72640 1 -- start start 2024-04-25T13:19:01.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f4337a72640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4330074930 0x7f433019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f4337a72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4330075f10 0x7f433019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f4337a72640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4330109e50 0x7f43301a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f4337a72640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f433010fd30 con 0x7f4330075f10 2024-04-25T13:19:01.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f4337a72640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f433010fbb0 con 0x7f4330074930 2024-04-25T13:19:01.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f4337a72640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f433010feb0 con 0x7f4330109e50 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f433500d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4330075f10 0x7f433019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f433500d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4330075f10 0x7f433019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60062/0 (socket says 172.21.15.31:60062) 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f433500d640 1 -- 172.21.15.31:0/3467567754 learned_addr learned my addr 172.21.15.31:0/3467567754 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f433600f640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4330109e50 0x7f43301a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f433580e640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4330074930 0x7f433019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f433500d640 1 -- 172.21.15.31:0/3467567754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4330109e50 msgr2=0x7f43301a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f433500d640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4330109e50 0x7f43301a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f433500d640 1 -- 172.21.15.31:0/3467567754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4330074930 msgr2=0x7f433019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f433500d640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4330074930 0x7f433019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.868+0000 7f433500d640 1 -- 172.21.15.31:0/3467567754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43301a20a0 con 0x7f4330075f10 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.869+0000 7f433500d640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4330075f10 0x7f433019a9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f432000b4a0 tx=0x7f432000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.869+0000 7f4326ffd640 1 -- 172.21.15.31:0/3467567754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320014070 con 0x7f4330075f10 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.869+0000 7f4326ffd640 1 -- 172.21.15.31:0/3467567754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4320010ac0 con 0x7f4330075f10 2024-04-25T13:19:01.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.869+0000 7f4337a72640 1 -- 172.21.15.31:0/3467567754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43301a2390 con 0x7f4330075f10 2024-04-25T13:19:01.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.869+0000 7f4326ffd640 1 -- 172.21.15.31:0/3467567754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320017c30 con 0x7f4330075f10 2024-04-25T13:19:01.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.869+0000 7f4337a72640 1 -- 172.21.15.31:0/3467567754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43301a2960 con 0x7f4330075f10 2024-04-25T13:19:01.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.870+0000 7f4337a72640 1 -- 172.21.15.31:0/3467567754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4330108c10 con 0x7f4330075f10 2024-04-25T13:19:01.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.871+0000 7f4326ffd640 1 -- 172.21.15.31:0/3467567754 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4320017070 con 0x7f4330075f10 2024-04-25T13:19:01.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.871+0000 7f4326ffd640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f431c0a3c40 0x7f431c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.872+0000 7f433580e640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f431c0a3c40 0x7f431c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.872+0000 7f4326ffd640 1 -- 172.21.15.31:0/3467567754 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f43200cfc30 con 0x7f4330075f10 2024-04-25T13:19:01.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.872+0000 7f433580e640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f431c0a3c40 0x7f431c0a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f432c000c00 tx=0x7f432c0314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:01.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.875+0000 7f4326ffd640 1 -- 172.21.15.31:0/3467567754 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f432009c540 con 0x7f4330075f10 2024-04-25T13:19:01.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:01.897+0000 7f4326ffd640 1 -- 172.21.15.31:0/3467567754 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4320010c70 con 0x7f4330075f10 2024-04-25T13:19:02.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.071+0000 7f4337a72640 1 -- 172.21.15.31:0/3467567754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f433002d810 con 0x7f4330075f10 2024-04-25T13:19:02.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.071+0000 7f4326ffd640 1 -- 172.21.15.31:0/3467567754 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f43200c9f40 con 0x7f4330075f10 2024-04-25T13:19:02.072 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T13:19:02.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 -- 172.21.15.31:0/3467567754 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f431c0a3c40 msgr2=0x7f431c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f431c0a3c40 0x7f431c0a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f432c000c00 tx=0x7f432c0314a0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 -- 172.21.15.31:0/3467567754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4330075f10 msgr2=0x7f433019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4330075f10 0x7f433019a9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f432000b4a0 tx=0x7f432000b960 comp rx=0 tx=0).stop 2024-04-25T13:19:02.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 -- 172.21.15.31:0/3467567754 shutdown_connections 2024-04-25T13:19:02.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f431c0a3c40 0x7f431c0a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4330109e50 0x7f43301a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4330075f10 0x7f433019a9c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 --2- 172.21.15.31:0/3467567754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4330074930 0x7f433019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 -- 172.21.15.31:0/3467567754 >> 172.21.15.31:0/3467567754 conn(0x7f43300fd530 msgr2=0x7f4330105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:02.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 -- 172.21.15.31:0/3467567754 shutdown_connections 2024-04-25T13:19:02.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.073+0000 7f4337a72640 1 -- 172.21.15.31:0/3467567754 wait complete. 2024-04-25T13:19:02.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:01 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/309309255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:02.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:01 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1144466970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:02.084 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:01 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3017387810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:02.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:19:02.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:01 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/309309255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:02.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:01 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1144466970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:02.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:01 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3017387810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.189+0000 7fb98154f640 1 -- 172.21.15.31:0/3230770094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb97c107d30 msgr2=0x7fb97c10a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.189+0000 7fb98154f640 1 --2- 172.21.15.31:0/3230770094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb97c107d30 0x7fb97c10a120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb974009ca0 tx=0x7fb97402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.189+0000 7fb98154f640 1 -- 172.21.15.31:0/3230770094 shutdown_connections 2024-04-25T13:19:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.189+0000 7fb98154f640 1 --2- 172.21.15.31:0/3230770094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb97c107d30 0x7fb97c10a120 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.189+0000 7fb98154f640 1 --2- 172.21.15.31:0/3230770094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb97c105400 0x7fb97c1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.189+0000 7fb98154f640 1 --2- 172.21.15.31:0/3230770094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb97c102ad0 0x7fb97c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.189+0000 7fb98154f640 1 -- 172.21.15.31:0/3230770094 >> 172.21.15.31:0/3230770094 conn(0x7fb97c0fc440 msgr2=0x7fb97c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.189+0000 7fb98154f640 1 -- 172.21.15.31:0/3230770094 shutdown_connections 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.189+0000 7fb98154f640 1 -- 172.21.15.31:0/3230770094 wait complete. 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb98154f640 1 Processor -- start 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb98154f640 1 -- start start 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb98154f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb97c102ad0 0x7fb97c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb98154f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb97c105400 0x7fb97c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb98154f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb97c107d30 0x7fb97c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb98154f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c10db50 con 0x7fb97c105400 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb98154f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c10d9d0 con 0x7fb97c107d30 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb98154f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c10dcd0 con 0x7fb97c102ad0 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb97c105400 0x7fb97c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:02.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb97c105400 0x7fb97c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60094/0 (socket says 172.21.15.31:60094) 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97a7fc640 1 -- 172.21.15.31:0/652270550 learned_addr learned my addr 172.21.15.31:0/652270550 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97b7fe640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb97c107d30 0x7fb97c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97affd640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb97c102ad0 0x7fb97c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97a7fc640 1 -- 172.21.15.31:0/652270550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb97c102ad0 msgr2=0x7fb97c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97a7fc640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb97c102ad0 0x7fb97c1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97a7fc640 1 -- 172.21.15.31:0/652270550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb97c107d30 msgr2=0x7fb97c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97a7fc640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb97c107d30 0x7fb97c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97a7fc640 1 -- 172.21.15.31:0/652270550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c19fde0 con 0x7fb97c105400 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97affd640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb97c102ad0 0x7fb97c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.190+0000 7fb97a7fc640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb97c105400 0x7fb97c198700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb97000d770 tx=0x7fb97000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.191+0000 7fb963fff640 1 -- 172.21.15.31:0/652270550 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970017070 con 0x7fb97c105400 2024-04-25T13:19:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.191+0000 7fb98154f640 1 -- 172.21.15.31:0/652270550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c1a00d0 con 0x7fb97c105400 2024-04-25T13:19:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.191+0000 7fb963fff640 1 -- 172.21.15.31:0/652270550 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb970009c30 con 0x7fb97c105400 2024-04-25T13:19:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.191+0000 7fb963fff640 1 -- 172.21.15.31:0/652270550 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97000f620 con 0x7fb97c105400 2024-04-25T13:19:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.191+0000 7fb98154f640 1 -- 172.21.15.31:0/652270550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c1a06f0 con 0x7fb97c105400 2024-04-25T13:19:02.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.192+0000 7fb98154f640 1 -- 172.21.15.31:0/652270550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97c0fdf30 con 0x7fb97c105400 2024-04-25T13:19:02.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.193+0000 7fb963fff640 1 -- 172.21.15.31:0/652270550 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb97001a070 con 0x7fb97c105400 2024-04-25T13:19:02.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.193+0000 7fb963fff640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb9640a3bb0 0x7fb9640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.194+0000 7fb97affd640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb9640a3bb0 0x7fb9640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:02.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.194+0000 7fb963fff640 1 -- 172.21.15.31:0/652270550 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fb9700cffd0 con 0x7fb97c105400 2024-04-25T13:19:02.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.194+0000 7fb97affd640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb9640a3bb0 0x7fb9640a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb96c006fd0 tx=0x7fb96c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:02.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.197+0000 7fb963fff640 1 -- 172.21.15.31:0/652270550 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb970009de0 con 0x7fb97c105400 2024-04-25T13:19:02.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:01 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/309309255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:02.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:01 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1144466970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:02.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:01 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3017387810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.383+0000 7fb98154f640 1 -- 172.21.15.31:0/652270550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb97c02d7c0 con 0x7fb97c105400 2024-04-25T13:19:02.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.383+0000 7fb963fff640 1 -- 172.21.15.31:0/652270550 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fb97009c990 con 0x7fb97c105400 2024-04-25T13:19:02.384 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T13:19:02.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 -- 172.21.15.31:0/652270550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb9640a3bb0 msgr2=0x7fb9640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb9640a3bb0 0x7fb9640a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb96c006fd0 tx=0x7fb96c008040 comp rx=0 tx=0).stop 2024-04-25T13:19:02.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 -- 172.21.15.31:0/652270550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb97c105400 msgr2=0x7fb97c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb97c105400 0x7fb97c198700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb97000d770 tx=0x7fb97000dc30 comp rx=0 tx=0).stop 2024-04-25T13:19:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 -- 172.21.15.31:0/652270550 shutdown_connections 2024-04-25T13:19:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb9640a3bb0 0x7fb9640a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb97c107d30 0x7fb97c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb97c105400 0x7fb97c198700 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 --2- 172.21.15.31:0/652270550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb97c102ad0 0x7fb97c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 -- 172.21.15.31:0/652270550 >> 172.21.15.31:0/652270550 conn(0x7fb97c0fc440 msgr2=0x7fb97c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 -- 172.21.15.31:0/652270550 shutdown_connections 2024-04-25T13:19:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.385+0000 7fb98154f640 1 -- 172.21.15.31:0/652270550 wait complete. 2024-04-25T13:19:02.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:19:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.501+0000 7f02dfe10640 1 -- 172.21.15.31:0/2823219207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d81053c0 msgr2=0x7f02d81077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.501+0000 7f02dfe10640 1 --2- 172.21.15.31:0/2823219207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d81053c0 0x7f02d81077b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009900 tx=0x7f02c802a410 comp rx=0 tx=0).stop 2024-04-25T13:19:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.501+0000 7f02dfe10640 1 -- 172.21.15.31:0/2823219207 shutdown_connections 2024-04-25T13:19:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.501+0000 7f02dfe10640 1 --2- 172.21.15.31:0/2823219207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d8107cf0 0x7f02d810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.501+0000 7f02dfe10640 1 --2- 172.21.15.31:0/2823219207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d81053c0 0x7f02d81077b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.501+0000 7f02dfe10640 1 --2- 172.21.15.31:0/2823219207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d8102a90 0x7f02d8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.501+0000 7f02dfe10640 1 -- 172.21.15.31:0/2823219207 >> 172.21.15.31:0/2823219207 conn(0x7f02d80fc440 msgr2=0x7f02d80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.501+0000 7f02dfe10640 1 -- 172.21.15.31:0/2823219207 shutdown_connections 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.501+0000 7f02dfe10640 1 -- 172.21.15.31:0/2823219207 wait complete. 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02dfe10640 1 Processor -- start 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02dfe10640 1 -- start start 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02dfe10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d8102a90 0x7f02d81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02dfe10640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d81053c0 0x7f02d8198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02dfe10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d8107cf0 0x7f02d819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02dfe10640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f02d810db30 con 0x7f02d8107cf0 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02dfe10640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f02d810d9b0 con 0x7f02d8102a90 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02dfe10640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f02d810dcb0 con 0x7f02d81053c0 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02de3ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d8107cf0 0x7f02d819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02de3ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d8107cf0 0x7f02d819f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60120/0 (socket says 172.21.15.31:60120) 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02de3ad640 1 -- 172.21.15.31:0/950811395 learned_addr learned my addr 172.21.15.31:0/950811395 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02dd3ab640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d81053c0 0x7f02d8198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02ddbac640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d8102a90 0x7f02d81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02de3ad640 1 -- 172.21.15.31:0/950811395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d81053c0 msgr2=0x7f02d8198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02de3ad640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d81053c0 0x7f02d8198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02de3ad640 1 -- 172.21.15.31:0/950811395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d8102a90 msgr2=0x7f02d81982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02de3ad640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d8102a90 0x7f02d81982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02de3ad640 1 -- 172.21.15.31:0/950811395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d819ff10 con 0x7f02d8107cf0 2024-04-25T13:19:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.502+0000 7f02de3ad640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d8107cf0 0x7f02d819f8a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f02d000cad0 tx=0x7f02d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.503+0000 7f02ceffd640 1 -- 172.21.15.31:0/950811395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d000ae20 con 0x7f02d8107cf0 2024-04-25T13:19:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.503+0000 7f02dfe10640 1 -- 172.21.15.31:0/950811395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d81a0200 con 0x7f02d8107cf0 2024-04-25T13:19:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.503+0000 7f02ceffd640 1 -- 172.21.15.31:0/950811395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d0005020 con 0x7f02d8107cf0 2024-04-25T13:19:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.503+0000 7f02ceffd640 1 -- 172.21.15.31:0/950811395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d000f6b0 con 0x7f02d8107cf0 2024-04-25T13:19:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.503+0000 7f02dfe10640 1 -- 172.21.15.31:0/950811395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d81a07a0 con 0x7f02d8107cf0 2024-04-25T13:19:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.504+0000 7f02dfe10640 1 -- 172.21.15.31:0/950811395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d80fdf30 con 0x7f02d8107cf0 2024-04-25T13:19:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.505+0000 7f02ceffd640 1 -- 172.21.15.31:0/950811395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f02d00051d0 con 0x7f02d8107cf0 2024-04-25T13:19:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.505+0000 7f02ceffd640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f02c40a3bb0 0x7f02c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.506+0000 7f02ddbac640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f02c40a3bb0 0x7f02c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.506+0000 7f02ceffd640 1 -- 172.21.15.31:0/950811395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f02d00d0590 con 0x7f02d8107cf0 2024-04-25T13:19:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.506+0000 7f02ddbac640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f02c40a3bb0 0x7f02c40a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009890 tx=0x7f02d4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:02.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.510+0000 7f02ceffd640 1 -- 172.21.15.31:0/950811395 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02d0014030 con 0x7f02d8107cf0 2024-04-25T13:19:02.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.696+0000 7f02dfe10640 1 -- 172.21.15.31:0/950811395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f02d8062b50 con 0x7f02d8107cf0 2024-04-25T13:19:02.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.696+0000 7f02ceffd640 1 -- 172.21.15.31:0/950811395 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f02d009cf50 con 0x7f02d8107cf0 2024-04-25T13:19:02.697 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.698+0000 7f02dfe10640 1 -- 172.21.15.31:0/950811395 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f02c40a3bb0 msgr2=0x7f02c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.698+0000 7f02dfe10640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f02c40a3bb0 0x7f02c40a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009890 tx=0x7f02d4008040 comp rx=0 tx=0).stop 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.698+0000 7f02dfe10640 1 -- 172.21.15.31:0/950811395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d8107cf0 msgr2=0x7f02d819f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.698+0000 7f02dfe10640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d8107cf0 0x7f02d819f8a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f02d000cad0 tx=0x7f02d000cf90 comp rx=0 tx=0).stop 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.698+0000 7f02dfe10640 1 -- 172.21.15.31:0/950811395 shutdown_connections 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.698+0000 7f02dfe10640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f02c40a3bb0 0x7f02c40a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.698+0000 7f02dfe10640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d8107cf0 0x7f02d819f8a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.698+0000 7f02dfe10640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d81053c0 0x7f02d8198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.698+0000 7f02dfe10640 1 --2- 172.21.15.31:0/950811395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02d8102a90 0x7f02d81982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.698+0000 7f02dfe10640 1 -- 172.21.15.31:0/950811395 >> 172.21.15.31:0/950811395 conn(0x7f02d80fc440 msgr2=0x7f02d8103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.699+0000 7f02dfe10640 1 -- 172.21.15.31:0/950811395 shutdown_connections 2024-04-25T13:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.699+0000 7f02dfe10640 1 -- 172.21.15.31:0/950811395 wait complete. 2024-04-25T13:19:02.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:19:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.810+0000 7fb4b4de2640 1 -- 172.21.15.31:0/550915297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4b01029a0 msgr2=0x7fb4b01092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.810+0000 7fb4b4de2640 1 --2- 172.21.15.31:0/550915297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4b01029a0 0x7fb4b01092c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009ca0 tx=0x7fb4a802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.811+0000 7fb4b4de2640 1 -- 172.21.15.31:0/550915297 shutdown_connections 2024-04-25T13:19:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.811+0000 7fb4b4de2640 1 --2- 172.21.15.31:0/550915297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4b01029a0 0x7fb4b01092c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.811+0000 7fb4b4de2640 1 --2- 172.21.15.31:0/550915297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b0101fe0 0x7fb4b0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.811+0000 7fb4b4de2640 1 --2- 172.21.15.31:0/550915297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b0100e90 0x7fb4b01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.811+0000 7fb4b4de2640 1 -- 172.21.15.31:0/550915297 >> 172.21.15.31:0/550915297 conn(0x7fb4b00fc400 msgr2=0x7fb4b00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.811+0000 7fb4b4de2640 1 -- 172.21.15.31:0/550915297 shutdown_connections 2024-04-25T13:19:02.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.811+0000 7fb4b4de2640 1 -- 172.21.15.31:0/550915297 wait complete. 2024-04-25T13:19:02.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.811+0000 7fb4b4de2640 1 Processor -- start 2024-04-25T13:19:02.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.811+0000 7fb4b4de2640 1 -- start start 2024-04-25T13:19:02.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4b4de2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4b0100e90 0x7fb4b0104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4b4de2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b0101fe0 0x7fb4b0103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4b4de2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b01029a0 0x7fb4b0103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4b4de2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b010c9f0 con 0x7fb4b0100e90 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4b4de2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b010c870 con 0x7fb4b01029a0 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4b4de2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b010cb70 con 0x7fb4b0101fe0 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4ae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4b0100e90 0x7fb4b0104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4ae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4b0100e90 0x7fb4b0104e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60130/0 (socket says 172.21.15.31:60130) 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4ae59c640 1 -- 172.21.15.31:0/4157107880 learned_addr learned my addr 172.21.15.31:0/4157107880 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4ae59c640 1 -- 172.21.15.31:0/4157107880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b0101fe0 msgr2=0x7fb4b0103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4ae59c640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b0101fe0 0x7fb4b0103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4ae59c640 1 -- 172.21.15.31:0/4157107880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b01029a0 msgr2=0x7fb4b0103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4ae59c640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b01029a0 0x7fb4b0103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4ae59c640 1 -- 172.21.15.31:0/4157107880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b0104220 con 0x7fb4b0100e90 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4ae59c640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4b0100e90 0x7fb4b0104e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000d770 tx=0x7fb4a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb49f7fe640 1 -- 172.21.15.31:0/4157107880 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0017070 con 0x7fb4b0100e90 2024-04-25T13:19:02.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb49f7fe640 1 -- 172.21.15.31:0/4157107880 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a00054d0 con 0x7fb4b0100e90 2024-04-25T13:19:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4b4de2640 1 -- 172.21.15.31:0/4157107880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b01a22e0 con 0x7fb4b0100e90 2024-04-25T13:19:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb49f7fe640 1 -- 172.21.15.31:0/4157107880 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0004c30 con 0x7fb4b0100e90 2024-04-25T13:19:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.812+0000 7fb4b4de2640 1 -- 172.21.15.31:0/4157107880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b01a27e0 con 0x7fb4b0100e90 2024-04-25T13:19:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.813+0000 7fb4b4de2640 1 -- 172.21.15.31:0/4157107880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b01a2aa0 con 0x7fb4b0100e90 2024-04-25T13:19:02.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.814+0000 7fb49f7fe640 1 -- 172.21.15.31:0/4157107880 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb4a0004e50 con 0x7fb4b0100e90 2024-04-25T13:19:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.815+0000 7fb49f7fe640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4980a3bb0 0x7fb4980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.815+0000 7fb4add9b640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4980a3bb0 0x7fb4980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.815+0000 7fb49f7fe640 1 -- 172.21.15.31:0/4157107880 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fb4a00cfff0 con 0x7fb4b0100e90 2024-04-25T13:19:02.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.815+0000 7fb4add9b640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4980a3bb0 0x7fb4980a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4005fd0 tx=0x7fb4a400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:02.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:02.820+0000 7fb49f7fe640 1 -- 172.21.15.31:0/4157107880 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4a009c990 con 0x7fb4b0100e90 2024-04-25T13:19:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:02 smithi031 ceph-mon[29457]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:02 smithi031 ceph-mon[29457]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:19:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:02 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3467567754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:02 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/652270550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:02 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/950811395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.048+0000 7fb4b4de2640 1 -- 172.21.15.31:0/4157107880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4b01a3210 con 0x7fb4b0100e90 2024-04-25T13:19:03.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.049+0000 7fb49f7fe640 1 -- 172.21.15.31:0/4157107880 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11798 (secure 0 0 0) 0x7fb4a009c290 con 0x7fb4b0100e90 2024-04-25T13:19:03.050 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:03.050 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2599371925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2599371925},{"type":"v1","addr":"172.21.15.31:6839","nonce":2599371925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/164386980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":164386980},{"type":"v1","addr":"172.21.15.31:6841","nonce":164386980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:17:40.162672+0000","modified":"2024-04-25T13:17:44.194940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2682191586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2682191586},{"type":"v1","addr":"172.21.15.66:6839","nonce":2682191586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24511]},"id":1}]} 2024-04-25T13:19:03.050 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-04-25T13:19:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 -- 172.21.15.31:0/4157107880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4980a3bb0 msgr2=0x7fb4980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4980a3bb0 0x7fb4980a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4005fd0 tx=0x7fb4a400b410 comp rx=0 tx=0).stop 2024-04-25T13:19:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 -- 172.21.15.31:0/4157107880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4b0100e90 msgr2=0x7fb4b0104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4b0100e90 0x7fb4b0104e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000d770 tx=0x7fb4a000dc30 comp rx=0 tx=0).stop 2024-04-25T13:19:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 -- 172.21.15.31:0/4157107880 shutdown_connections 2024-04-25T13:19:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4980a3bb0 0x7fb4980a6070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4b01029a0 0x7fb4b0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b0101fe0 0x7fb4b0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 --2- 172.21.15.31:0/4157107880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4b0100e90 0x7fb4b0104e10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 -- 172.21.15.31:0/4157107880 >> 172.21.15.31:0/4157107880 conn(0x7fb4b00fc400 msgr2=0x7fb4b00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 -- 172.21.15.31:0/4157107880 shutdown_connections 2024-04-25T13:19:03.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.051+0000 7fb4b4de2640 1 -- 172.21.15.31:0/4157107880 wait complete. 2024-04-25T13:19:03.063 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:19:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:02 smithi066 ceph-mon[33137]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:02 smithi066 ceph-mon[33137]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:19:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:02 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3467567754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:02 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/652270550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:02 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/950811395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.164+0000 7f0878a8a640 1 -- 172.21.15.31:0/1759280930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0874102a30 msgr2=0x7f0874104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.164+0000 7f0878a8a640 1 --2- 172.21.15.31:0/1759280930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0874102a30 0x7f0874104e20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0864009900 tx=0x7f086402a410 comp rx=0 tx=0).stop 2024-04-25T13:19:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.164+0000 7f0878a8a640 1 -- 172.21.15.31:0/1759280930 shutdown_connections 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.164+0000 7f0878a8a640 1 --2- 172.21.15.31:0/1759280930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0874107c90 0x7f087410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.164+0000 7f0878a8a640 1 --2- 172.21.15.31:0/1759280930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0874105360 0x7f0874107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.164+0000 7f0878a8a640 1 --2- 172.21.15.31:0/1759280930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0874102a30 0x7f0874104e20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.164+0000 7f0878a8a640 1 -- 172.21.15.31:0/1759280930 >> 172.21.15.31:0/1759280930 conn(0x7f08740fc440 msgr2=0x7f08740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.164+0000 7f0878a8a640 1 -- 172.21.15.31:0/1759280930 shutdown_connections 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.164+0000 7f0878a8a640 1 -- 172.21.15.31:0/1759280930 wait complete. 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.165+0000 7f0878a8a640 1 Processor -- start 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.165+0000 7f0878a8a640 1 -- start start 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.165+0000 7f0878a8a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0874102a30 0x7f0874198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.165+0000 7f0878a8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0874105360 0x7f08741987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.165+0000 7f0878a8a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0874107c90 0x7f087419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.165+0000 7f0878a8a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f087410daf0 con 0x7f0874105360 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.165+0000 7f0878a8a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f087410d970 con 0x7f0874107c90 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.165+0000 7f0878a8a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f087410dc70 con 0x7f0874102a30 2024-04-25T13:19:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.165+0000 7f0871d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0874105360 0x7f08741987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.165+0000 7f0871d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0874105360 0x7f08741987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60146/0 (socket says 172.21.15.31:60146) 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.165+0000 7f0871d9b640 1 -- 172.21.15.31:0/3077380689 learned_addr learned my addr 172.21.15.31:0/3077380689 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f087259c640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0874102a30 0x7f0874198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f0871d9b640 1 -- 172.21.15.31:0/3077380689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0874102a30 msgr2=0x7f0874198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f0871d9b640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0874102a30 0x7f0874198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f0872d9d640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0874107c90 0x7f087419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f0871d9b640 1 -- 172.21.15.31:0/3077380689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0874107c90 msgr2=0x7f087419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f0871d9b640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0874107c90 0x7f087419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f0871d9b640 1 -- 172.21.15.31:0/3077380689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087419fe40 con 0x7f0874105360 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f0872d9d640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0874107c90 0x7f087419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f0871d9b640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0874105360 0x7f08741987b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0868005fd0 tx=0x7f086800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f08637fe640 1 -- 172.21.15.31:0/3077380689 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868017070 con 0x7f0874105360 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f08637fe640 1 -- 172.21.15.31:0/3077380689 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0868009c30 con 0x7f0874105360 2024-04-25T13:19:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f0878a8a640 1 -- 172.21.15.31:0/3077380689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08741a0130 con 0x7f0874105360 2024-04-25T13:19:03.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f08637fe640 1 -- 172.21.15.31:0/3077380689 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868003d30 con 0x7f0874105360 2024-04-25T13:19:03.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.166+0000 7f0878a8a640 1 -- 172.21.15.31:0/3077380689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08741a06d0 con 0x7f0874105360 2024-04-25T13:19:03.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.167+0000 7f0878a8a640 1 -- 172.21.15.31:0/3077380689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08740fdf30 con 0x7f0874105360 2024-04-25T13:19:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.168+0000 7f08637fe640 1 -- 172.21.15.31:0/3077380689 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0868003ed0 con 0x7f0874105360 2024-04-25T13:19:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.169+0000 7f08637fe640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f085c0a3c30 0x7f085c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.169+0000 7f087259c640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f085c0a3c30 0x7f085c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.169+0000 7f08637fe640 1 -- 172.21.15.31:0/3077380689 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f08680d0180 con 0x7f0874105360 2024-04-25T13:19:03.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.170+0000 7f087259c640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f085c0a3c30 0x7f085c0a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0864000c00 tx=0x7f08640314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:03.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.173+0000 7f08637fe640 1 -- 172.21.15.31:0/3077380689 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0868020030 con 0x7f0874105360 2024-04-25T13:19:03.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:02 smithi136 ceph-mon[33038]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:02 smithi136 ceph-mon[33038]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:19:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:02 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3467567754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:02 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/652270550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:02 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/950811395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.359+0000 7f0878a8a640 1 -- 172.21.15.31:0/3077380689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0874062b50 con 0x7f0874105360 2024-04-25T13:19:03.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.361+0000 7f08637fe640 1 -- 172.21.15.31:0/3077380689 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22650 (secure 0 0 0) 0x7f0868009de0 con 0x7f0874105360 2024-04-25T13:19:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:03.362 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:18:32.938284+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.2e","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 -- 172.21.15.31:0/3077380689 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f085c0a3c30 msgr2=0x7f085c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f085c0a3c30 0x7f085c0a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0864000c00 tx=0x7f08640314a0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 -- 172.21.15.31:0/3077380689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0874105360 msgr2=0x7f08741987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0874105360 0x7f08741987b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0868005fd0 tx=0x7f086800dde0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 -- 172.21.15.31:0/3077380689 shutdown_connections 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f085c0a3c30 0x7f085c0a60f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0874107c90 0x7f087419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0874105360 0x7f08741987b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 --2- 172.21.15.31:0/3077380689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0874102a30 0x7f0874198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 -- 172.21.15.31:0/3077380689 >> 172.21.15.31:0/3077380689 conn(0x7f08740fc440 msgr2=0x7f08741034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 -- 172.21.15.31:0/3077380689 shutdown_connections 2024-04-25T13:19:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.363+0000 7f0878a8a640 1 -- 172.21.15.31:0/3077380689 wait complete. 2024-04-25T13:19:03.376 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:19:03.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.476+0000 7f41e6519640 1 -- 172.21.15.31:0/777581907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e0107c90 msgr2=0x7f41e010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.476+0000 7f41e6519640 1 --2- 172.21.15.31:0/777581907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e0107c90 0x7f41e010a0e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009ca0 tx=0x7f41d802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.476+0000 7f41e6519640 1 -- 172.21.15.31:0/777581907 shutdown_connections 2024-04-25T13:19:03.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.476+0000 7f41e6519640 1 --2- 172.21.15.31:0/777581907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e0107c90 0x7f41e010a0e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.476+0000 7f41e6519640 1 --2- 172.21.15.31:0/777581907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0105360 0x7f41e0107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.476+0000 7f41e6519640 1 --2- 172.21.15.31:0/777581907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e0102a30 0x7f41e0104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.476+0000 7f41e6519640 1 -- 172.21.15.31:0/777581907 >> 172.21.15.31:0/777581907 conn(0x7f41e00fc440 msgr2=0x7f41e00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:03.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.476+0000 7f41e6519640 1 -- 172.21.15.31:0/777581907 shutdown_connections 2024-04-25T13:19:03.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.476+0000 7f41e6519640 1 -- 172.21.15.31:0/777581907 wait complete. 2024-04-25T13:19:03.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e6519640 1 Processor -- start 2024-04-25T13:19:03.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e6519640 1 -- start start 2024-04-25T13:19:03.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e6519640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e0102a30 0x7f41e01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e6519640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0105360 0x7f41e0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e6519640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e0107c90 0x7f41e019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e6519640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f41e010daf0 con 0x7f41e0107c90 2024-04-25T13:19:03.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e6519640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41e010d970 con 0x7f41e0105360 2024-04-25T13:19:03.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e6519640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f41e010dc70 con 0x7f41e0102a30 2024-04-25T13:19:03.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e4ab6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e0107c90 0x7f41e019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e4ab6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e0107c90 0x7f41e019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60158/0 (socket says 172.21.15.31:60158) 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e4ab6640 1 -- 172.21.15.31:0/1799873554 learned_addr learned my addr 172.21.15.31:0/1799873554 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41df7fe640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0105360 0x7f41e0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41dffff640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e0102a30 0x7f41e01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e4ab6640 1 -- 172.21.15.31:0/1799873554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e0102a30 msgr2=0x7f41e01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e4ab6640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e0102a30 0x7f41e01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e4ab6640 1 -- 172.21.15.31:0/1799873554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0105360 msgr2=0x7f41e0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e4ab6640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0105360 0x7f41e0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41e4ab6640 1 -- 172.21.15.31:0/1799873554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e019fde0 con 0x7f41e0107c90 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41dffff640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e0102a30 0x7f41e01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.477+0000 7f41df7fe640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0105360 0x7f41e0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.478+0000 7f41e4ab6640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e0107c90 0x7f41e019f770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f41d80097b0 tx=0x7f41d8031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.478+0000 7f41dd7fa640 1 -- 172.21.15.31:0/1799873554 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8003b70 con 0x7f41e0107c90 2024-04-25T13:19:03.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.478+0000 7f41dd7fa640 1 -- 172.21.15.31:0/1799873554 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41d8003d10 con 0x7f41e0107c90 2024-04-25T13:19:03.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.478+0000 7f41e6519640 1 -- 172.21.15.31:0/1799873554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e01a0070 con 0x7f41e0107c90 2024-04-25T13:19:03.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.478+0000 7f41dd7fa640 1 -- 172.21.15.31:0/1799873554 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d802fc10 con 0x7f41e0107c90 2024-04-25T13:19:03.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.478+0000 7f41e6519640 1 -- 172.21.15.31:0/1799873554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e01a05c0 con 0x7f41e0107c90 2024-04-25T13:19:03.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.479+0000 7f41e6519640 1 -- 172.21.15.31:0/1799873554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e00fdf30 con 0x7f41e0107c90 2024-04-25T13:19:03.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.480+0000 7f41dd7fa640 1 -- 172.21.15.31:0/1799873554 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f41d8004040 con 0x7f41e0107c90 2024-04-25T13:19:03.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.480+0000 7f41dd7fa640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f41c80a3bb0 0x7f41c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.481+0000 7f41dffff640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f41c80a3bb0 0x7f41c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.481+0000 7f41dd7fa640 1 -- 172.21.15.31:0/1799873554 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f41d80f1b20 con 0x7f41e0107c90 2024-04-25T13:19:03.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.481+0000 7f41dffff640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f41c80a3bb0 0x7f41c80a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009b50 tx=0x7f41d0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:03.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.485+0000 7f41dd7fa640 1 -- 172.21.15.31:0/1799873554 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41d80be480 con 0x7f41e0107c90 2024-04-25T13:19:03.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.711+0000 7f41e6519640 1 -- 172.21.15.31:0/1799873554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41e0062b50 con 0x7f41e0107c90 2024-04-25T13:19:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.712+0000 7f41dd7fa640 1 -- 172.21.15.31:0/1799873554 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11798 (secure 0 0 0) 0x7f41d80bdd80 con 0x7f41e0107c90 2024-04-25T13:19:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:03.713 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2599371925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2599371925},{"type":"v1","addr":"172.21.15.31:6839","nonce":2599371925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/164386980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":164386980},{"type":"v1","addr":"172.21.15.31:6841","nonce":164386980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:17:40.162672+0000","modified":"2024-04-25T13:17:44.194940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2682191586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2682191586},{"type":"v1","addr":"172.21.15.66:6839","nonce":2682191586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24511]},"id":1}]} 2024-04-25T13:19:03.714 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-04-25T13:19:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 -- 172.21.15.31:0/1799873554 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f41c80a3bb0 msgr2=0x7f41c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f41c80a3bb0 0x7f41c80a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009b50 tx=0x7f41d0005f90 comp rx=0 tx=0).stop 2024-04-25T13:19:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 -- 172.21.15.31:0/1799873554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e0107c90 msgr2=0x7f41e019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e0107c90 0x7f41e019f770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f41d80097b0 tx=0x7f41d8031770 comp rx=0 tx=0).stop 2024-04-25T13:19:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 -- 172.21.15.31:0/1799873554 shutdown_connections 2024-04-25T13:19:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f41c80a3bb0 0x7f41c80a6070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e0107c90 0x7f41e019f770 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41e0105360 0x7f41e0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 --2- 172.21.15.31:0/1799873554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e0102a30 0x7f41e01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 -- 172.21.15.31:0/1799873554 >> 172.21.15.31:0/1799873554 conn(0x7f41e00fc440 msgr2=0x7f41e0103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 -- 172.21.15.31:0/1799873554 shutdown_connections 2024-04-25T13:19:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.714+0000 7f41e6519640 1 -- 172.21.15.31:0/1799873554 wait complete. 2024-04-25T13:19:03.727 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:17:40.162672+0000', 'modified': '2024-04-25T13:17:44.194940+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24511}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6839/2682191586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 2682191586}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 2682191586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.136:6839/2196424437', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 2196424437}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 2196424437}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24511, 'qdb_cluster': [24511]} 2024-04-25T13:19:03.727 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:19:03.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:19:03.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.829+0000 7f1f864ef640 1 -- 172.21.15.31:0/1679273979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f80102440 msgr2=0x7f1f80102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.829+0000 7f1f864ef640 1 --2- 172.21.15.31:0/1679273979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f80102440 0x7f1f80102860 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009900 tx=0x7f1f7002a410 comp rx=0 tx=0).stop 2024-04-25T13:19:03.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.829+0000 7f1f864ef640 1 -- 172.21.15.31:0/1679273979 shutdown_connections 2024-04-25T13:19:03.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.829+0000 7f1f864ef640 1 --2- 172.21.15.31:0/1679273979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f80101b30 0x7f1f801093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.829+0000 7f1f864ef640 1 --2- 172.21.15.31:0/1679273979 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f80101170 0x7f1f801015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.829+0000 7f1f864ef640 1 --2- 172.21.15.31:0/1679273979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f80102440 0x7f1f80102860 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.829+0000 7f1f864ef640 1 -- 172.21.15.31:0/1679273979 >> 172.21.15.31:0/1679273979 conn(0x7f1f800fbbc0 msgr2=0x7f1f800fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:03.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.829+0000 7f1f864ef640 1 -- 172.21.15.31:0/1679273979 shutdown_connections 2024-04-25T13:19:03.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.829+0000 7f1f864ef640 1 -- 172.21.15.31:0/1679273979 wait complete. 2024-04-25T13:19:03.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.829+0000 7f1f864ef640 1 Processor -- start 2024-04-25T13:19:03.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.829+0000 7f1f864ef640 1 -- start start 2024-04-25T13:19:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f864ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f80101170 0x7f1f80198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f864ef640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f80101b30 0x7f1f80198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f864ef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f80102440 0x7f1f8019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f864ef640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8010dad0 con 0x7f1f80102440 2024-04-25T13:19:03.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f864ef640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8010d950 con 0x7f1f80101170 2024-04-25T13:19:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f864ef640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8010dc50 con 0x7f1f80101b30 2024-04-25T13:19:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f84a8c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f80102440 0x7f1f8019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f84a8c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f80102440 0x7f1f8019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60162/0 (socket says 172.21.15.31:60162) 2024-04-25T13:19:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f84a8c640 1 -- 172.21.15.31:0/3561521124 learned_addr learned my addr 172.21.15.31:0/3561521124 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f7ffff640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f80101170 0x7f1f80198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f80101b30 0x7f1f80198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f84a8c640 1 -- 172.21.15.31:0/3561521124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f80101b30 msgr2=0x7f1f80198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f84a8c640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f80101b30 0x7f1f80198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f84a8c640 1 -- 172.21.15.31:0/3561521124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f80101170 msgr2=0x7f1f80198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f84a8c640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f80101170 0x7f1f80198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f84a8c640 1 -- 172.21.15.31:0/3561521124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8019fe20 con 0x7f1f80102440 2024-04-25T13:19:03.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f80101b30 0x7f1f80198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:03.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f84a8c640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f80102440 0x7f1f8019f7b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800eaa0 tx=0x7f1f7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:03.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.830+0000 7f1f7d7fa640 1 -- 172.21.15.31:0/3561521124 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800cdb0 con 0x7f1f80102440 2024-04-25T13:19:03.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.831+0000 7f1f7d7fa640 1 -- 172.21.15.31:0/3561521124 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f78005020 con 0x7f1f80102440 2024-04-25T13:19:03.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.831+0000 7f1f7d7fa640 1 -- 172.21.15.31:0/3561521124 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f780116d0 con 0x7f1f80102440 2024-04-25T13:19:03.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.831+0000 7f1f864ef640 1 -- 172.21.15.31:0/3561521124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f801a0110 con 0x7f1f80102440 2024-04-25T13:19:03.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.831+0000 7f1f864ef640 1 -- 172.21.15.31:0/3561521124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f801a0610 con 0x7f1f80102440 2024-04-25T13:19:03.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.833+0000 7f1f7d7fa640 1 -- 172.21.15.31:0/3561521124 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1f780051d0 con 0x7f1f80102440 2024-04-25T13:19:03.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.833+0000 7f1f89207740 1 Processor -- start 2024-04-25T13:19:03.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.833+0000 7f1f89207740 1 -- start start 2024-04-25T13:19:03.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.833+0000 7f1f89207740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed518b80 0x5648ed518fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.833+0000 7f1f89207740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed51a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.833+0000 7f1f89207740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed51a710 0x5648ed520fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.833+0000 7f1f89207740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5648ed47dc50 con 0x5648ed518b80 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.833+0000 7f1f89207740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5648ed41a830 con 0x5648ed519d50 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.833+0000 7f1f89207740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5648ed479c10 con 0x5648ed51a710 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.833+0000 7f1f7f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed518b80 0x5648ed518fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f7f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed518b80 0x5648ed518fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60178/0 (socket says 172.21.15.31:60178) 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/349468652 learned_addr learned my addr 172.21.15.31:0/349468652 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f7ffff640 1 --2- 172.21.15.31:0/349468652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed51a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f7d7fa640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f680a3bb0 0x7f1f680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/349468652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed51a710 msgr2=0x5648ed520fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/349468652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed51a710 0x5648ed520fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/349468652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 msgr2=0x5648ed51a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/349468652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed51a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/349468652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ed1902d0 con 0x5648ed518b80 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f84a8c640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f680a3bb0 0x7f1f680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f7d7fa640 1 -- 172.21.15.31:0/3561521124 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f1f780d0680 con 0x7f1f80102440 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/349468652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed518b80 0x5648ed518fa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400ccf0 tx=0x7f1f74010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7940186ad94040e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f667fc640 1 -- 172.21.15.31:0/349468652 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74016070 con 0x5648ed518b80 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f667fc640 1 -- 172.21.15.31:0/349468652 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f740047a0 con 0x5648ed518b80 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.834+0000 7f1f667fc640 1 -- 172.21.15.31:0/349468652 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74012800 con 0x5648ed518b80 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f89207740 1 -- 172.21.15.31:0/349468652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed518b80 msgr2=0x5648ed518fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f84a8c640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f680a3bb0 0x7f1f680a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1f78098d40 tx=0x7f1f7806e370 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f89207740 1 --2- 172.21.15.31:0/349468652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed518b80 0x5648ed518fa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400ccf0 tx=0x7f1f74010740 comp rx=0 tx=0).stop 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f89207740 1 -- 172.21.15.31:0/349468652 shutdown_connections 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f89207740 1 --2- 172.21.15.31:0/349468652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed51a710 0x5648ed520fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f89207740 1 --2- 172.21.15.31:0/349468652 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed51a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f89207740 1 --2- 172.21.15.31:0/349468652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed518b80 0x5648ed518fa0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f89207740 1 -- 172.21.15.31:0/349468652 >> 172.21.15.31:0/349468652 conn(0x5648ed514070 msgr2=0x5648ed514470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f89207740 1 -- 172.21.15.31:0/349468652 shutdown_connections 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f89207740 1 -- 172.21.15.31:0/349468652 wait complete. 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f89207740 1 -- start start 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.835+0000 7f1f89207740 1 Processor -- start 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f89207740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed518b80 0x5648ed523030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f89207740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed523570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f89207740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed52a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f89207740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5648ed521720 con 0x5648ed51a710 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f89207740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5648ed418470 con 0x5648ed519d50 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f89207740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5648ed5218a0 con 0x5648ed518b80 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f89207740 10 client.0 _ll_register_callbacks cb 0x5648ed43df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f7f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed52a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f7f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed52a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60188/0 (socket says 172.21.15.31:60188) 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/304701416 learned_addr learned my addr 172.21.15.31:0/304701416 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed518b80 0x5648ed523030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f7ffff640 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed523570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/304701416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed518b80 msgr2=0x5648ed523030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed518b80 0x5648ed523030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/304701416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 msgr2=0x5648ed523570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed523570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/304701416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ed25a6b0 con 0x5648ed51a710 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed52a5b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1f740058c0 tx=0x7f1f74004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405946cfc90bef15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/304701416 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74016040 con 0x5648ed51a710 2024-04-25T13:19:03.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/304701416 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f1f74004be0 con 0x5648ed51a710 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.836+0000 7f1f89207740 1 -- 172.21.15.31:0/304701416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f740047a0 con 0x5648ed51a710 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/304701416 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1f74005000 con 0x5648ed51a710 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f89207740 10 client.15048 fetch_fsmap learned FSMap version 20 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f89207740 1 -- 172.21.15.31:0/304701416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5648ed52d970 con 0x5648ed51a710 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/304701416 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f740047a0 con 0x5648ed51a710 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/304701416 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1f7401a990 con 0x5648ed51a710 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f89207740 10 client.15048 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f89207740 10 client.15048 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f89207740 20 client.15048 populate_metadata read hostname 'smithi031' 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f89207740 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5648ed52fcd0 0x5648ed531140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f89207740 4 client.15048 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f89207740 1 -- 172.21.15.31:0/304701416 --> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648ed48b670 con 0x5648ed52fcd0 2024-04-25T13:19:03.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.837+0000 7f1f7ffff640 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5648ed52fcd0 0x5648ed531140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.838+0000 7f1f5f7fe640 10 client.15048 ms_handle_connect on v2:172.21.15.136:6838/2196424437 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.838+0000 7f1f7ffff640 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5648ed52fcd0 0x5648ed531140 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35afae5ceb02b840 server_cookie=9f7c5612b9ff09a1 in_seq=0 out_seq=0 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.838+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/304701416 <== mds.0 v2:172.21.15.136:6838/2196424437 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648ed48b670 con 0x5648ed52fcd0 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.838+0000 7f1f5f7fe640 10 client.15048 handle_command_reply: tid=0 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 1 client.15048 shutdown 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 1 -- 172.21.15.31:0/304701416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 msgr2=0x5648ed52a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed52a5b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1f740058c0 tx=0x7f1f74004e90 comp rx=0 tx=0).stop 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 1 -- 172.21.15.31:0/304701416 shutdown_connections 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5648ed52fcd0 0x5648ed531140 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed52a5b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed523570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f5f7fe640 0 client.15048 ms_handle_reset on v2:172.21.15.136:6838/2196424437 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 1 --2- 172.21.15.31:0/304701416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed518b80 0x5648ed523030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 1 -- 172.21.15.31:0/304701416 >> 172.21.15.31:0/304701416 conn(0x5648ed514070 msgr2=0x5648ed5131d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:03.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 1 -- 172.21.15.31:0/304701416 shutdown_connections 2024-04-25T13:19:03.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 1 -- 172.21.15.31:0/304701416 wait complete. 2024-04-25T13:19:03.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.839+0000 7f1f89207740 20 client.15048 trim_cache size 0 max 16384 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.857+0000 7f1f89207740 1 Processor -- start 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.857+0000 7f1f89207740 1 -- start start 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.857+0000 7f1f89207740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed518b80 0x5648ed5234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.857+0000 7f1f89207740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed52a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.857+0000 7f1f89207740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed520eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.857+0000 7f1f89207740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5648ed52f480 con 0x5648ed51a710 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.857+0000 7f1f89207740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5648ed52fb50 con 0x5648ed519d50 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.857+0000 7f1f89207740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5648ed530220 con 0x5648ed518b80 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.857+0000 7f1f7f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed520eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.857+0000 7f1f7f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed520eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60190/0 (socket says 172.21.15.31:60190) 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.857+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/2215104361 learned_addr learned my addr 172.21.15.31:0/2215104361 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:03.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.858+0000 7f1f7ffff640 1 --2- 172.21.15.31:0/2215104361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed52a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.858+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/2215104361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed518b80 msgr2=0x5648ed5234d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.858+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/2215104361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed518b80 0x5648ed5234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.858+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/2215104361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 msgr2=0x5648ed52a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.858+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/2215104361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed52a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.858+0000 7f1f7f7fe640 1 -- 172.21.15.31:0/2215104361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ed5197d0 con 0x5648ed51a710 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.858+0000 7f1f7ffff640 1 --2- 172.21.15.31:0/2215104361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed52a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.858+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/2215104361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed520eb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005ac0 tx=0x7f1f7400c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0db452893e0f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.858+0000 7f1f65ffb640 1 -- 172.21.15.31:0/2215104361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74039070 con 0x5648ed51a710 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.858+0000 7f1f65ffb640 1 -- 172.21.15.31:0/2215104361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f74004780 con 0x5648ed51a710 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.858+0000 7f1f65ffb640 1 -- 172.21.15.31:0/2215104361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74016040 con 0x5648ed51a710 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 -- 172.21.15.31:0/2215104361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 msgr2=0x5648ed520eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 --2- 172.21.15.31:0/2215104361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed520eb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005ac0 tx=0x7f1f7400c620 comp rx=0 tx=0).stop 2024-04-25T13:19:03.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 -- 172.21.15.31:0/2215104361 shutdown_connections 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 --2- 172.21.15.31:0/2215104361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed51a710 0x5648ed520eb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 --2- 172.21.15.31:0/2215104361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed519d50 0x5648ed52a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 --2- 172.21.15.31:0/2215104361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed518b80 0x5648ed5234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 -- 172.21.15.31:0/2215104361 >> 172.21.15.31:0/2215104361 conn(0x5648ed514070 msgr2=0x5648ed52ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 -- 172.21.15.31:0/2215104361 shutdown_connections 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 -- 172.21.15.31:0/2215104361 wait complete. 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 -- start start 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 Processor -- start 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed518b80 0x5648ed525ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed519d50 0x5648ed525ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed51a710 0x5648ed529010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5648ed530fc0 con 0x5648ed519d50 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5648ed5308f0 con 0x5648ed518b80 2024-04-25T13:19:03.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.859+0000 7f1f89207740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5648ed531690 con 0x5648ed51a710 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f89207740 10 client.0 _ll_register_callbacks cb 0x7f1f7400cbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f7ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed519d50 0x5648ed525ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f7ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed519d50 0x5648ed525ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60204/0 (socket says 172.21.15.31:60204) 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f7ffff640 1 -- 172.21.15.31:0/2265484356 learned_addr learned my addr 172.21.15.31:0/2265484356 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed51a710 0x5648ed529010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f7ffff640 1 -- 172.21.15.31:0/2265484356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed51a710 msgr2=0x5648ed529010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f7ffff640 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed51a710 0x5648ed529010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f7ffff640 1 -- 172.21.15.31:0/2265484356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed518b80 msgr2=0x5648ed525ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f7ffff640 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed518b80 0x5648ed525ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f7ffff640 1 -- 172.21.15.31:0/2265484356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ed529600 con 0x5648ed519d50 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed51a710 0x5648ed529010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f7ffff640 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed519d50 0x5648ed525ff0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009900 tx=0x7f1f70031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b1939063d95a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T13:19:03.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/2265484356 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7004a060 con 0x5648ed519d50 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/2265484356 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f1f70031990 con 0x5648ed519d50 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f89207740 1 -- 172.21.15.31:0/2265484356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f74004780 con 0x5648ed519d50 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/2265484356 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1f70035040 con 0x5648ed519d50 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f89207740 10 client.15054 fetch_fsmap learned FSMap version 20 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f89207740 1 -- 172.21.15.31:0/2265484356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5648ed52a910 con 0x5648ed519d50 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.860+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/2265484356 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f74004780 con 0x5648ed519d50 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/2265484356 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1f70003680 con 0x5648ed519d50 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f89207740 10 client.15054 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f89207740 10 client.15054 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f89207740 20 client.15054 populate_metadata read hostname 'smithi031' 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f89207740 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5648ed59a600 0x5648ed599310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f89207740 4 client.15054 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f89207740 1 -- 172.21.15.31:0/2265484356 --> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] -- command(tid 0: {"prefix": "status"}) -- 0x5648ed4854e0 con 0x5648ed59a600 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5648ed59a600 0x5648ed599310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:03.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f7f7fe640 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5648ed59a600 0x5648ed599310 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f0bc2475df1e35 server_cookie=2cd3b2c3c9ac684a in_seq=0 out_seq=0 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f5f7fe640 10 client.15054 ms_handle_connect on v2:172.21.15.136:6838/2196424437 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f5f7fe640 1 -- 172.21.15.31:0/2265484356 <== mds.0 v2:172.21.15.136:6838/2196424437 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5648ed4854e0 con 0x5648ed59a600 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.861+0000 7f1f5f7fe640 10 client.15054 handle_command_reply: tid=0 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 1 client.15054 shutdown 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 1 -- 172.21.15.31:0/2265484356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed519d50 msgr2=0x5648ed525ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed519d50 0x5648ed525ff0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009900 tx=0x7f1f70031920 comp rx=0 tx=0).stop 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 1 -- 172.21.15.31:0/2265484356 shutdown_connections 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5648ed59a600 0x5648ed599310 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648ed51a710 0x5648ed529010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f5f7fe640 0 client.15054 ms_handle_reset on v2:172.21.15.136:6838/2196424437 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648ed519d50 0x5648ed525ff0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 1 --2- 172.21.15.31:0/2265484356 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648ed518b80 0x5648ed525ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 1 -- 172.21.15.31:0/2265484356 >> 172.21.15.31:0/2265484356 conn(0x5648ed514070 msgr2=0x5648ed514450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 1 -- 172.21.15.31:0/2265484356 shutdown_connections 2024-04-25T13:19:03.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 1 -- 172.21.15.31:0/2265484356 wait complete. 2024-04-25T13:19:03.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f89207740 20 client.15054 trim_cache size 0 max 16384 2024-04-25T13:19:03.870 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T13:19:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "e4b023f4-0304-11ef-bc93-c7b262605968", 2024-04-25T13:19:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-04-25T13:19:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24511, 2024-04-25T13:19:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T13:19:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T13:19:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T13:19:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 83.062081775999999, 2024-04-25T13:19:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 20, 2024-04-25T13:19:03.870 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 90, 2024-04-25T13:19:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T13:19:03.871 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 96.665507387999995 2024-04-25T13:19:03.871 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T13:19:03.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f864ef640 1 -- 172.21.15.31:0/3561521124 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f680a3bb0 msgr2=0x7f1f680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f864ef640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f680a3bb0 0x7f1f680a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1f78098d40 tx=0x7f1f7806e370 comp rx=0 tx=0).stop 2024-04-25T13:19:03.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f864ef640 1 -- 172.21.15.31:0/3561521124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f80102440 msgr2=0x7f1f8019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:03.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.862+0000 7f1f864ef640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f80102440 0x7f1f8019f7b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800eaa0 tx=0x7f1f7800ef60 comp rx=0 tx=0).stop 2024-04-25T13:19:03.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.863+0000 7f1f864ef640 1 -- 172.21.15.31:0/3561521124 shutdown_connections 2024-04-25T13:19:03.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.863+0000 7f1f864ef640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f1f680a3bb0 0x7f1f680a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.863+0000 7f1f864ef640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f80102440 0x7f1f8019f7b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.863+0000 7f1f864ef640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f80101b30 0x7f1f80198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.863+0000 7f1f864ef640 1 --2- 172.21.15.31:0/3561521124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f80101170 0x7f1f80198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:03.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.863+0000 7f1f864ef640 1 -- 172.21.15.31:0/3561521124 >> 172.21.15.31:0/3561521124 conn(0x7f1f800fbbc0 msgr2=0x7f1f80105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:03.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.863+0000 7f1f864ef640 1 -- 172.21.15.31:0/3561521124 shutdown_connections 2024-04-25T13:19:03.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:03.863+0000 7f1f864ef640 1 -- 172.21.15.31:0/3561521124 wait complete. 2024-04-25T13:19:03.874 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T13:19:03.874 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:19:03.884 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T13:19:03.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T13:19:03.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:03 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/4157107880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:03.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:03 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3077380689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:03.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:03 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1799873554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:04.128 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:19:04.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:03 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/4157107880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:04.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:03 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3077380689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:04.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:03 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1799873554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:04.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:03 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/4157107880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:04.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:03 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3077380689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:04.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:03 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1799873554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:05.138 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:04 smithi031 ceph-mon[29457]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:19:05.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:04 smithi066 ceph-mon[33137]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:19:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:04 smithi136 ceph-mon[33038]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:19:05.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.648+0000 7efe55275700 1 -- 172.21.15.31:0/2598951242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe50104640 msgr2=0x7efe5010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:05.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.648+0000 7efe55275700 1 --2- 172.21.15.31:0/2598951242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe50104640 0x7efe5010aef0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe4800b0d0 tx=0x7efe4800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:19:05.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.648+0000 7efe55275700 1 -- 172.21.15.31:0/2598951242 shutdown_connections 2024-04-25T13:19:05.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.648+0000 7efe55275700 1 --2- 172.21.15.31:0/2598951242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe50102aa0 0x7efe50102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:05.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.648+0000 7efe55275700 1 --2- 172.21.15.31:0/2598951242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe50104640 0x7efe5010aef0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:05.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.648+0000 7efe55275700 1 --2- 172.21.15.31:0/2598951242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe50103ca0 0x7efe50104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:05.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.648+0000 7efe55275700 1 -- 172.21.15.31:0/2598951242 >> 172.21.15.31:0/2598951242 conn(0x7efe500fe040 msgr2=0x7efe50100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:05.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.648+0000 7efe55275700 1 -- 172.21.15.31:0/2598951242 shutdown_connections 2024-04-25T13:19:05.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.648+0000 7efe55275700 1 -- 172.21.15.31:0/2598951242 wait complete. 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.648+0000 7efe55275700 1 Processor -- start 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe55275700 1 -- start start 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe55275700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe50102aa0 0x7efe500767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe55275700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe50103ca0 0x7efe50074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe55275700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe50104640 0x7efe50075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe55275700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efe5010e540 con 0x7efe50102aa0 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe55275700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efe5010e3c0 con 0x7efe50104640 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe55275700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efe5010e6c0 con 0x7efe50103ca0 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe4effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe50102aa0 0x7efe500767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe4effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe50102aa0 0x7efe500767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60230/0 (socket says 172.21.15.31:60230) 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe4effd700 1 -- 172.21.15.31:0/2773162722 learned_addr learned my addr 172.21.15.31:0/2773162722 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe4f7fe700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe50104640 0x7efe50075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe4e7fc700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe50103ca0 0x7efe50074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe4effd700 1 -- 172.21.15.31:0/2773162722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe50103ca0 msgr2=0x7efe50074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe4effd700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe50103ca0 0x7efe50074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe4effd700 1 -- 172.21.15.31:0/2773162722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe50104640 msgr2=0x7efe50075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe4effd700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe50104640 0x7efe50075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe4effd700 1 -- 172.21.15.31:0/2773162722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe50075c20 con 0x7efe50102aa0 2024-04-25T13:19:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.649+0000 7efe4effd700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe50102aa0 0x7efe500767e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c8f0 tx=0x7efe4000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.650+0000 7efe37fff700 1 -- 172.21.15.31:0/2773162722 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40004220 con 0x7efe50102aa0 2024-04-25T13:19:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.650+0000 7efe55275700 1 -- 172.21.15.31:0/2773162722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe501a2d10 con 0x7efe50102aa0 2024-04-25T13:19:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.650+0000 7efe37fff700 1 -- 172.21.15.31:0/2773162722 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe400043c0 con 0x7efe50102aa0 2024-04-25T13:19:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.650+0000 7efe37fff700 1 -- 172.21.15.31:0/2773162722 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40003c60 con 0x7efe50102aa0 2024-04-25T13:19:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.650+0000 7efe55275700 1 -- 172.21.15.31:0/2773162722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe501a3170 con 0x7efe50102aa0 2024-04-25T13:19:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.651+0000 7efe55275700 1 -- 172.21.15.31:0/2773162722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5002df80 con 0x7efe50102aa0 2024-04-25T13:19:05.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.652+0000 7efe37fff700 1 -- 172.21.15.31:0/2773162722 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efe40007c80 con 0x7efe50102aa0 2024-04-25T13:19:05.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.652+0000 7efe37fff700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efe380a3c40 0x7efe380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:05.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.652+0000 7efe4e7fc700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efe380a3c40 0x7efe380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:05.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.652+0000 7efe37fff700 1 -- 172.21.15.31:0/2773162722 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7efe400d0390 con 0x7efe50102aa0 2024-04-25T13:19:05.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.653+0000 7efe4e7fc700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efe380a3c40 0x7efe380a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efe44005950 tx=0x7efe440058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:05.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.656+0000 7efe37fff700 1 -- 172.21.15.31:0/2773162722 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe40014030 con 0x7efe50102aa0 2024-04-25T13:19:05.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.885+0000 7efe55275700 1 -- 172.21.15.31:0/2773162722 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7efe50062e80 con 0x7efe380a3c40 2024-04-25T13:19:05.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.886+0000 7efe37fff700 1 -- 172.21.15.31:0/2773162722 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7efe50062e80 con 0x7efe380a3c40 2024-04-25T13:19:05.886 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2024-04-25T13:19:05.886 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2024-04-25T13:19:05.886 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2024-04-25T13:19:05.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.888+0000 7efe55275700 1 -- 172.21.15.31:0/2773162722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efe380a3c40 msgr2=0x7efe380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:05.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.888+0000 7efe55275700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efe380a3c40 0x7efe380a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efe44005950 tx=0x7efe440058e0 comp rx=0 tx=0).stop 2024-04-25T13:19:05.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.888+0000 7efe55275700 1 -- 172.21.15.31:0/2773162722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe50102aa0 msgr2=0x7efe500767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:05.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.888+0000 7efe55275700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe50102aa0 0x7efe500767e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c8f0 tx=0x7efe4000cc00 comp rx=0 tx=0).stop 2024-04-25T13:19:05.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.888+0000 7efe55275700 1 -- 172.21.15.31:0/2773162722 shutdown_connections 2024-04-25T13:19:05.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.888+0000 7efe55275700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efe50104640 0x7efe50075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:05.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.888+0000 7efe55275700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe50102aa0 0x7efe500767e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:05.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.888+0000 7efe55275700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7efe380a3c40 0x7efe380a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:05.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.888+0000 7efe55275700 1 --2- 172.21.15.31:0/2773162722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe50103ca0 0x7efe50074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:05.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.888+0000 7efe55275700 1 -- 172.21.15.31:0/2773162722 >> 172.21.15.31:0/2773162722 conn(0x7efe500fe040 msgr2=0x7efe50107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:05.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.889+0000 7efe55275700 1 -- 172.21.15.31:0/2773162722 shutdown_connections 2024-04-25T13:19:05.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:05.889+0000 7efe55275700 1 -- 172.21.15.31:0/2773162722 wait complete. 2024-04-25T13:19:06.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T13:19:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:19:07.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:06 smithi066 ceph-mon[33137]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:19:07.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:06 smithi066 ceph-mon[33137]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:07.198 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:06 smithi031 ceph-mon[29457]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:19:07.198 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:06 smithi031 ceph-mon[29457]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:07.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:06 smithi136 ceph-mon[33038]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:19:07.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:06 smithi136 ceph-mon[33038]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.392+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2337315393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e54103cb0 msgr2=0x7f2e54104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.392+0000 7f2e5b4a3700 1 --2- 172.21.15.31:0/2337315393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e54103cb0 0x7f2e54104110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2e500099c0 tx=0x7f2e50009cd0 comp rx=0 tx=0).stop 2024-04-25T13:19:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.392+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2337315393 shutdown_connections 2024-04-25T13:19:08.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.392+0000 7f2e5b4a3700 1 --2- 172.21.15.31:0/2337315393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e54102ab0 0x7f2e54102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.392+0000 7f2e5b4a3700 1 --2- 172.21.15.31:0/2337315393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e54104650 0x7f2e5410af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.392+0000 7f2e5b4a3700 1 --2- 172.21.15.31:0/2337315393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e54103cb0 0x7f2e54104110 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.392+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2337315393 >> 172.21.15.31:0/2337315393 conn(0x7f2e540fe050 msgr2=0x7f2e54100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.392+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2337315393 shutdown_connections 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.392+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2337315393 wait complete. 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5b4a3700 1 Processor -- start 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5b4a3700 1 -- start start 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5b4a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e54102ab0 0x7f2e54198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5b4a3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e54103cb0 0x7f2e541991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5b4a3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e54104650 0x7f2e541a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5b4a3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5410e5c0 con 0x7f2e54102ab0 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5b4a3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5410e440 con 0x7f2e54103cb0 2024-04-25T13:19:08.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5b4a3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5410e740 con 0x7f2e54104650 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5923f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e54102ab0 0x7f2e54198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5923f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e54102ab0 0x7f2e54198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60256/0 (socket says 172.21.15.31:60256) 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5923f700 1 -- 172.21.15.31:0/2861220772 learned_addr learned my addr 172.21.15.31:0/2861220772 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e58a3e700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e54103cb0 0x7f2e541991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e59a40700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e54104650 0x7f2e541a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5923f700 1 -- 172.21.15.31:0/2861220772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e54104650 msgr2=0x7f2e541a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5923f700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e54104650 0x7f2e541a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5923f700 1 -- 172.21.15.31:0/2861220772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e54103cb0 msgr2=0x7f2e541991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5923f700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e54103cb0 0x7f2e541991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.393+0000 7f2e5923f700 1 -- 172.21.15.31:0/2861220772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e541a08c0 con 0x7f2e54102ab0 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.394+0000 7f2e5923f700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e54102ab0 0x7f2e54198c70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800b500 tx=0x7f2e4800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:08.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.394+0000 7f2e467fc700 1 -- 172.21.15.31:0/2861220772 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48015020 con 0x7f2e54102ab0 2024-04-25T13:19:08.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.394+0000 7f2e467fc700 1 -- 172.21.15.31:0/2861220772 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e48011ce0 con 0x7f2e54102ab0 2024-04-25T13:19:08.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.394+0000 7f2e467fc700 1 -- 172.21.15.31:0/2861220772 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48010d20 con 0x7f2e54102ab0 2024-04-25T13:19:08.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.394+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2861220772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e541a0bb0 con 0x7f2e54102ab0 2024-04-25T13:19:08.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.394+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2861220772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e541a10f0 con 0x7f2e54102ab0 2024-04-25T13:19:08.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.395+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2861220772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e54069730 con 0x7f2e54102ab0 2024-04-25T13:19:08.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.396+0000 7f2e467fc700 1 -- 172.21.15.31:0/2861220772 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2e48018020 con 0x7f2e54102ab0 2024-04-25T13:19:08.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.397+0000 7f2e467fc700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2e400a3c40 0x7f2e400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:08.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.397+0000 7f2e58a3e700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2e400a3c40 0x7f2e400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:08.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.397+0000 7f2e467fc700 1 -- 172.21.15.31:0/2861220772 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f2e480d00e0 con 0x7f2e54102ab0 2024-04-25T13:19:08.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.397+0000 7f2e58a3e700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2e400a3c40 0x7f2e400a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009990 tx=0x7f2e5000b910 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:08.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.400+0000 7f2e467fc700 1 -- 172.21.15.31:0/2861220772 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e4801f020 con 0x7f2e54102ab0 2024-04-25T13:19:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.620+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2861220772 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2e54062e80 con 0x7f2e400a3c40 2024-04-25T13:19:08.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.635+0000 7f2e467fc700 1 -- 172.21.15.31:0/2861220772 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f2e54062e80 con 0x7f2e400a3c40 2024-04-25T13:19:08.635 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T13:19:08.635 INFO:teuthology.orchestra.run.smithi031.stdout:mds.a smithi031 running (96s) 14s ago 95s 13.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8dcf8d9c8c91 2024-04-25T13:19:08.635 INFO:teuthology.orchestra.run.smithi031.stdout:mds.b smithi066 running (93s) 27s ago 92s 17.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3ff34147d10f 2024-04-25T13:19:08.635 INFO:teuthology.orchestra.run.smithi031.stdout:mds.c smithi136 running (107s) 27s ago 107s 16.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3653f2f06733 2024-04-25T13:19:08.635 INFO:teuthology.orchestra.run.smithi031.stdout:mds.d smithi031 running (115s) 14s ago 115s 15.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 930113d5b4b6 2024-04-25T13:19:08.635 INFO:teuthology.orchestra.run.smithi031.stdout:mds.e smithi066 running (112s) 27s ago 112s 14.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 549fb8454996 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mds.f smithi136 running (110s) 27s ago 109s 15.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2e9a08925b53 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mds.g smithi031 running (99s) 14s ago 98s 13.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f21d230cf931 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mds.h smithi066 running (104s) 27s ago 104s 15.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 b9d5d6c3c08b 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mds.i smithi136 running (101s) 27s ago 101s 25.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 7ba737bdf5b1 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mds.j smithi031 running (118s) 14s ago 118s 13.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 b246bde6ed8e 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mds.k smithi066 running (2m) 27s ago 2m 16.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 cb5e6fdeda39 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mds.l smithi136 running (2m) 27s ago 2m 14.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 dc2e90317663 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.x smithi031 *:9283,8765 running (9m) 14s ago 9m 515M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8f9a4fffd858 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.y smithi066 *:8443,8765 running (7m) 27s ago 7m 451M - 19.0.0-3156-gd69f13d8 da4b004c09e8 13a7b580e9ef 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.z smithi136 *:8443,8765 running (7m) 27s ago 7m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2d8062f9d8e8 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a smithi031 running (9m) 14s ago 9m 50.4M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 ab96c980d7cc 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b smithi066 running (7m) 27s ago 7m 42.8M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 f9fc2a020c46 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c smithi136 running (7m) 27s ago 7m 39.3M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 730fc5357bb7 2024-04-25T13:19:08.636 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 14s ago 7m 48.8M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 2d533a8dd4ae 2024-04-25T13:19:08.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi066 running (6m) 27s ago 6m 53.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 f88a5db2c0c6 2024-04-25T13:19:08.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi136 running (6m) 27s ago 6m 51.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 41d5fe0510ac 2024-04-25T13:19:08.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi031 running (5m) 14s ago 5m 53.7M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 1ea286ad3e4b 2024-04-25T13:19:08.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi066 running (5m) 27s ago 5m 54.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 9558c3c86573 2024-04-25T13:19:08.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi136 running (4m) 27s ago 4m 58.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 3a99c1bee720 2024-04-25T13:19:08.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 smithi031 running (4m) 14s ago 4m 54.1M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 15d2f28bd13c 2024-04-25T13:19:08.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 smithi066 running (4m) 27s ago 4m 46.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 1d2bc6b414c5 2024-04-25T13:19:08.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 smithi136 running (3m) 27s ago 3m 53.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 2855554da34a 2024-04-25T13:19:08.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 smithi031 running (3m) 14s ago 3m 50.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 918cc8e8e56f 2024-04-25T13:19:08.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 smithi066 running (2m) 27s ago 2m 53.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 32c4c522f4bf 2024-04-25T13:19:08.638 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 smithi136 running (2m) 27s ago 2m 45.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 41aa8a6db116 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2861220772 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2e400a3c40 msgr2=0x7f2e400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2e400a3c40 0x7f2e400a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009990 tx=0x7f2e5000b910 comp rx=0 tx=0).stop 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2861220772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e54102ab0 msgr2=0x7f2e54198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e54102ab0 0x7f2e54198c70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800b500 tx=0x7f2e4800b810 comp rx=0 tx=0).stop 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2861220772 shutdown_connections 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2e400a3c40 0x7f2e400a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e54104650 0x7f2e541a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e54102ab0 0x7f2e54198c70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 --2- 172.21.15.31:0/2861220772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e54103cb0 0x7f2e541991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2861220772 >> 172.21.15.31:0/2861220772 conn(0x7f2e540fe050 msgr2=0x7f2e54107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2861220772 shutdown_connections 2024-04-25T13:19:08.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:08.638+0000 7f2e5b4a3700 1 -- 172.21.15.31:0/2861220772 wait complete. 2024-04-25T13:19:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:08 smithi066 ceph-mon[33137]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:19:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:08 smithi136 ceph-mon[33038]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:19:09.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:08 smithi031 ceph-mon[29457]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:19:09.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T13:19:09.516 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:19:10.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:09 smithi136 ceph-mon[33038]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:10.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:09 smithi031 ceph-mon[29457]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:10.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:09 smithi066 ceph-mon[33137]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:11.004 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:10 smithi031 ceph-mon[29457]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:19:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.045+0000 7fc258b10700 1 -- 172.21.15.31:0/2043796344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2541013a0 msgr2=0x7fc2541017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.045+0000 7fc258b10700 1 --2- 172.21.15.31:0/2043796344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2541013a0 0x7fc2541017c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc244009990 tx=0x7fc244009ca0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.045+0000 7fc258b10700 1 -- 172.21.15.31:0/2043796344 shutdown_connections 2024-04-25T13:19:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.045+0000 7fc258b10700 1 --2- 172.21.15.31:0/2043796344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2541026c0 0x7fc254102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.045+0000 7fc258b10700 1 --2- 172.21.15.31:0/2043796344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2541013a0 0x7fc2541017c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.045+0000 7fc258b10700 1 --2- 172.21.15.31:0/2043796344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc254101d00 0x7fc254102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.045+0000 7fc258b10700 1 -- 172.21.15.31:0/2043796344 >> 172.21.15.31:0/2043796344 conn(0x7fc2540fcf40 msgr2=0x7fc2540ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.045+0000 7fc258b10700 1 -- 172.21.15.31:0/2043796344 shutdown_connections 2024-04-25T13:19:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.045+0000 7fc258b10700 1 -- 172.21.15.31:0/2043796344 wait complete. 2024-04-25T13:19:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.046+0000 7fc258b10700 1 Processor -- start 2024-04-25T13:19:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.046+0000 7fc258b10700 1 -- start start 2024-04-25T13:19:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.046+0000 7fc258b10700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2541013a0 0x7fc25419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc258b10700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc254101d00 0x7fc25419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc258b10700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2541026c0 0x7fc2541a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc258b10700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc2541107c0 con 0x7fc2541013a0 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc258b10700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc254110640 con 0x7fc2541026c0 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc258b10700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc254110940 con 0x7fc254101d00 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc25259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2541013a0 0x7fc25419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc25259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2541013a0 0x7fc25419aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60266/0 (socket says 172.21.15.31:60266) 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc25259c700 1 -- 172.21.15.31:0/1021169354 learned_addr learned my addr 172.21.15.31:0/1021169354 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc251d9b700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc254101d00 0x7fc25419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc252d9d700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2541026c0 0x7fc2541a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc25259c700 1 -- 172.21.15.31:0/1021169354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc254101d00 msgr2=0x7fc25419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc25259c700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc254101d00 0x7fc25419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc25259c700 1 -- 172.21.15.31:0/1021169354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2541026c0 msgr2=0x7fc2541a2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc25259c700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2541026c0 0x7fc2541a2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc25259c700 1 -- 172.21.15.31:0/1021169354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2541a2b30 con 0x7fc2541013a0 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc25259c700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2541013a0 0x7fc25419aed0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc244000c00 tx=0x7fc2440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc2437fe700 1 -- 172.21.15.31:0/1021169354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc244027070 con 0x7fc2541013a0 2024-04-25T13:19:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc2437fe700 1 -- 172.21.15.31:0/1021169354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc244004000 con 0x7fc2541013a0 2024-04-25T13:19:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc258b10700 1 -- 172.21.15.31:0/1021169354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2541a2dc0 con 0x7fc2541013a0 2024-04-25T13:19:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc2437fe700 1 -- 172.21.15.31:0/1021169354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc244022da0 con 0x7fc2541013a0 2024-04-25T13:19:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.047+0000 7fc258b10700 1 -- 172.21.15.31:0/1021169354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2541a31c0 con 0x7fc2541013a0 2024-04-25T13:19:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.048+0000 7fc258b10700 1 -- 172.21.15.31:0/1021169354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc25402df80 con 0x7fc2541013a0 2024-04-25T13:19:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.049+0000 7fc2437fe700 1 -- 172.21.15.31:0/1021169354 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc24402a070 con 0x7fc2541013a0 2024-04-25T13:19:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.050+0000 7fc2437fe700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc23c0a3c40 0x7fc23c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.050+0000 7fc2437fe700 1 -- 172.21.15.31:0/1021169354 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fc2440d16c0 con 0x7fc2541013a0 2024-04-25T13:19:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.050+0000 7fc251d9b700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc23c0a3c40 0x7fc23c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.051+0000 7fc251d9b700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc23c0a3c40 0x7fc23c0a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc248009a10 tx=0x7fc248009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.054+0000 7fc2437fe700 1 -- 172.21.15.31:0/1021169354 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc244031070 con 0x7fc2541013a0 2024-04-25T13:19:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:10 smithi066 ceph-mon[33137]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:19:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:10 smithi136 ceph-mon[33038]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:19:11.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.271+0000 7fc258b10700 1 -- 172.21.15.31:0/1021169354 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fc254062e80 con 0x7fc23c0a3c40 2024-04-25T13:19:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.277+0000 7fc2437fe700 1 -- 172.21.15.31:0/1021169354 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fc254062e80 con 0x7fc23c0a3c40 2024-04-25T13:19:11.278 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T13:19:11.278 INFO:teuthology.orchestra.run.smithi031.stdout:mds.all 12/12 29s ago 2m smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi066=b;smithi066=e;smithi066=h;smithi066=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-04-25T13:19:11.278 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 3/3 29s ago 7m smithi031=x;smithi066=y;smithi136=z;count:3 2024-04-25T13:19:11.278 INFO:teuthology.orchestra.run.smithi031.stdout:mon 3/3 29s ago 7m smithi031:172.21.15.31=a;smithi066:172.21.15.66=b;smithi136:172.21.15.136=c;count:3 2024-04-25T13:19:11.278 INFO:teuthology.orchestra.run.smithi031.stdout:osd 12 29s ago - 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.279+0000 7fc258b10700 1 -- 172.21.15.31:0/1021169354 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc23c0a3c40 msgr2=0x7fc23c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.279+0000 7fc258b10700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc23c0a3c40 0x7fc23c0a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc248009a10 tx=0x7fc248009190 comp rx=0 tx=0).stop 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.279+0000 7fc258b10700 1 -- 172.21.15.31:0/1021169354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2541013a0 msgr2=0x7fc25419aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.279+0000 7fc258b10700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2541013a0 0x7fc25419aed0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc244000c00 tx=0x7fc2440047b0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.279+0000 7fc258b10700 1 -- 172.21.15.31:0/1021169354 shutdown_connections 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.279+0000 7fc258b10700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2541026c0 0x7fc2541a2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.279+0000 7fc258b10700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2541013a0 0x7fc25419aed0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.280+0000 7fc258b10700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fc23c0a3c40 0x7fc23c0a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.280+0000 7fc258b10700 1 --2- 172.21.15.31:0/1021169354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc254101d00 0x7fc25419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.280+0000 7fc258b10700 1 -- 172.21.15.31:0/1021169354 >> 172.21.15.31:0/1021169354 conn(0x7fc2540fcf40 msgr2=0x7fc2540fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.280+0000 7fc258b10700 1 -- 172.21.15.31:0/1021169354 shutdown_connections 2024-04-25T13:19:11.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:11.280+0000 7fc258b10700 1 -- 172.21.15.31:0/1021169354 wait complete. 2024-04-25T13:19:11.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T13:19:12.141 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:19:12.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:11 smithi136 ceph-mon[33038]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:12.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:11 smithi031 ceph-mon[29457]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:12.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:11 smithi066 ceph-mon[33137]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:13.173 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:12 smithi031 ceph-mon[29457]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:19:13.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:12 smithi136 ceph-mon[33038]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:19:13.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:12 smithi066 ceph-mon[33137]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:19:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.658+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/850136724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd4103cc0 msgr2=0x7f2dd4104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.658+0000 7f2ddaf9c700 1 --2- 172.21.15.31:0/850136724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd4103cc0 0x7f2dd4104120 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00099c0 tx=0x7f2dc0009cd0 comp rx=0 tx=0).stop 2024-04-25T13:19:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/850136724 shutdown_connections 2024-04-25T13:19:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 --2- 172.21.15.31:0/850136724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dd4104660 0x7f2dd410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 --2- 172.21.15.31:0/850136724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dd4102ac0 0x7f2dd4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:13.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 --2- 172.21.15.31:0/850136724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd4103cc0 0x7f2dd4104120 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/850136724 >> 172.21.15.31:0/850136724 conn(0x7f2dd40fe060 msgr2=0x7f2dd41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/850136724 shutdown_connections 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/850136724 wait complete. 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 Processor -- start 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 -- start start 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dd4102ac0 0x7f2dd419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd4103cc0 0x7f2dd419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.659+0000 7f2ddaf9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dd4104660 0x7f2dd41a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2ddaf9c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410e5d0 con 0x7f2dd4103cc0 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2ddaf9c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410e450 con 0x7f2dd4104660 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2ddaf9c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410e750 con 0x7f2dd4102ac0 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd4103cc0 0x7f2dd419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd4103cc0 0x7f2dd419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45918/0 (socket says 172.21.15.31:45918) 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd3fff700 1 -- 172.21.15.31:0/1485622361 learned_addr learned my addr 172.21.15.31:0/1485622361 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd8d38700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dd4102ac0 0x7f2dd419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd9539700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dd4104660 0x7f2dd41a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd3fff700 1 -- 172.21.15.31:0/1485622361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dd4102ac0 msgr2=0x7f2dd419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd3fff700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dd4102ac0 0x7f2dd419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd3fff700 1 -- 172.21.15.31:0/1485622361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dd4104660 msgr2=0x7f2dd41a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd3fff700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dd4104660 0x7f2dd41a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd3fff700 1 -- 172.21.15.31:0/1485622361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd41a2a80 con 0x7f2dd4103cc0 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd3fff700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd4103cc0 0x7f2dd419b360 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005910 tx=0x7f2dc0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd1ffb700 1 -- 172.21.15.31:0/1485622361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0027070 con 0x7f2dd4103cc0 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd1ffb700 1 -- 172.21.15.31:0/1485622361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dc0022470 con 0x7f2dd4103cc0 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/1485622361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd41a2d10 con 0x7f2dd4103cc0 2024-04-25T13:19:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.660+0000 7f2dd1ffb700 1 -- 172.21.15.31:0/1485622361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc000f6c0 con 0x7f2dd4103cc0 2024-04-25T13:19:13.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.661+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/1485622361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd41a31a0 con 0x7f2dd4103cc0 2024-04-25T13:19:13.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.662+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/1485622361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd4069730 con 0x7f2dd4103cc0 2024-04-25T13:19:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.663+0000 7f2dd1ffb700 1 -- 172.21.15.31:0/1485622361 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2dc002a070 con 0x7f2dd4103cc0 2024-04-25T13:19:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.663+0000 7f2dd1ffb700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2dc40a3c40 0x7f2dc40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.664+0000 7f2dd8d38700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2dc40a3c40 0x7f2dc40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.664+0000 7f2dd1ffb700 1 -- 172.21.15.31:0/1485622361 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f2dc00d12b0 con 0x7f2dd4103cc0 2024-04-25T13:19:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.664+0000 7f2dd8d38700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2dc40a3c40 0x7f2dc40a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4103b20 tx=0x7f2dc8009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:13.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.668+0000 7f2dd1ffb700 1 -- 172.21.15.31:0/1485622361 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2dc005eff0 con 0x7f2dd4103cc0 2024-04-25T13:19:13.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.884+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/1485622361 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2dd4076390 con 0x7f2dc40a3c40 2024-04-25T13:19:13.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.885+0000 7f2dd1ffb700 1 -- 172.21.15.31:0/1485622361 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f2dd4076390 con 0x7f2dc40a3c40 2024-04-25T13:19:13.885 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2024-04-25T13:19:13.885 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2024-04-25T13:19:13.885 INFO:teuthology.orchestra.run.smithi031.stdout:smithi066 172.21.15.66 2024-04-25T13:19:13.886 INFO:teuthology.orchestra.run.smithi031.stdout:smithi136 172.21.15.136 2024-04-25T13:19:13.886 INFO:teuthology.orchestra.run.smithi031.stdout:3 hosts in cluster 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/1485622361 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2dc40a3c40 msgr2=0x7f2dc40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2dc40a3c40 0x7f2dc40a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4103b20 tx=0x7f2dc8009190 comp rx=0 tx=0).stop 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/1485622361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd4103cc0 msgr2=0x7f2dd419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd4103cc0 0x7f2dd419b360 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005910 tx=0x7f2dc0005e80 comp rx=0 tx=0).stop 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/1485622361 shutdown_connections 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f2dc40a3c40 0x7f2dc40a6100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dd4102ac0 0x7f2dd419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2dd4104660 0x7f2dd41a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 --2- 172.21.15.31:0/1485622361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2dd4103cc0 0x7f2dd419b360 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/1485622361 >> 172.21.15.31:0/1485622361 conn(0x7f2dd40fe060 msgr2=0x7f2dd4074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/1485622361 shutdown_connections 2024-04-25T13:19:13.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:13.888+0000 7f2ddaf9c700 1 -- 172.21.15.31:0/1485622361 wait complete. 2024-04-25T13:19:14.452 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T13:19:14.703 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:19:15.108 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:14 smithi031 ceph-mon[29457]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:19:15.109 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:14 smithi031 ceph-mon[29457]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:15.210 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:14 smithi066 ceph-mon[33137]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:19:15.210 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:14 smithi066 ceph-mon[33137]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:15.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:14 smithi136 ceph-mon[33038]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:19:15.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:14 smithi136 ceph-mon[33038]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:16.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.359+0000 7fa957102700 1 -- 172.21.15.31:0/3165319353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9501019c0 msgr2=0x7fa950101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:16.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.359+0000 7fa957102700 1 --2- 172.21.15.31:0/3165319353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9501019c0 0x7fa950101de0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa9400099c0 tx=0x7fa940009cd0 comp rx=0 tx=0).stop 2024-04-25T13:19:16.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.359+0000 7fa957102700 1 -- 172.21.15.31:0/3165319353 shutdown_connections 2024-04-25T13:19:16.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.359+0000 7fa957102700 1 --2- 172.21.15.31:0/3165319353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa950103560 0x7fa950109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:16.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.359+0000 7fa957102700 1 --2- 172.21.15.31:0/3165319353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9501019c0 0x7fa950101de0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:16.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.359+0000 7fa957102700 1 --2- 172.21.15.31:0/3165319353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa950102bc0 0x7fa950103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:16.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.359+0000 7fa957102700 1 -- 172.21.15.31:0/3165319353 >> 172.21.15.31:0/3165319353 conn(0x7fa9500fcf60 msgr2=0x7fa9500ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:16.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.359+0000 7fa957102700 1 -- 172.21.15.31:0/3165319353 shutdown_connections 2024-04-25T13:19:16.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.359+0000 7fa957102700 1 -- 172.21.15.31:0/3165319353 wait complete. 2024-04-25T13:19:16.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.360+0000 7fa957102700 1 Processor -- start 2024-04-25T13:19:16.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.360+0000 7fa957102700 1 -- start start 2024-04-25T13:19:16.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.360+0000 7fa957102700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9501019c0 0x7fa95019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:16.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.360+0000 7fa957102700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa950102bc0 0x7fa95019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:16.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.360+0000 7fa957102700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa950103560 0x7fa9501a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:16.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.360+0000 7fa957102700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa95010d4b0 con 0x7fa950103560 2024-04-25T13:19:16.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.360+0000 7fa957102700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa95010d330 con 0x7fa950102bc0 2024-04-25T13:19:16.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.360+0000 7fa957102700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa95010d630 con 0x7fa9501019c0 2024-04-25T13:19:16.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa95569f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa950103560 0x7fa9501a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:16.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa95569f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa950103560 0x7fa9501a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45934/0 (socket says 172.21.15.31:45934) 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa95569f700 1 -- 172.21.15.31:0/514664393 learned_addr learned my addr 172.21.15.31:0/514664393 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa954e9e700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9501019c0 0x7fa95019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa94ffff700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa950102bc0 0x7fa95019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa95569f700 1 -- 172.21.15.31:0/514664393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9501019c0 msgr2=0x7fa95019aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa95569f700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9501019c0 0x7fa95019aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa95569f700 1 -- 172.21.15.31:0/514664393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa950102bc0 msgr2=0x7fa95019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa95569f700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa950102bc0 0x7fa95019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa95569f700 1 -- 172.21.15.31:0/514664393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9501a2a80 con 0x7fa950103560 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa94ffff700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa950102bc0 0x7fa95019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa95569f700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa950103560 0x7fa9501a23e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa94800b9a0 tx=0x7fa94800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa94dffb700 1 -- 172.21.15.31:0/514664393 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948019070 con 0x7fa950103560 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.361+0000 7fa957102700 1 -- 172.21.15.31:0/514664393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9501a2d70 con 0x7fa950103560 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.362+0000 7fa94dffb700 1 -- 172.21.15.31:0/514664393 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa948014440 con 0x7fa950103560 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.362+0000 7fa94dffb700 1 -- 172.21.15.31:0/514664393 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94800ce60 con 0x7fa950103560 2024-04-25T13:19:16.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.362+0000 7fa957102700 1 -- 172.21.15.31:0/514664393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9501a3270 con 0x7fa950103560 2024-04-25T13:19:16.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.363+0000 7fa957102700 1 -- 172.21.15.31:0/514664393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa950069760 con 0x7fa950103560 2024-04-25T13:19:16.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.364+0000 7fa94dffb700 1 -- 172.21.15.31:0/514664393 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa94801c070 con 0x7fa950103560 2024-04-25T13:19:16.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.364+0000 7fa94dffb700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa9380a3cc0 0x7fa9380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:16.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.365+0000 7fa94dffb700 1 -- 172.21.15.31:0/514664393 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fa9480cf760 con 0x7fa950103560 2024-04-25T13:19:16.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.365+0000 7fa954e9e700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa9380a3cc0 0x7fa9380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:16.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.365+0000 7fa954e9e700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa9380a3cc0 0x7fa9380a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa94000b580 tx=0x7fa940005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.369+0000 7fa94dffb700 1 -- 172.21.15.31:0/514664393 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa950069760 con 0x7fa950103560 2024-04-25T13:19:16.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.585+0000 7fa957102700 1 -- 172.21.15.31:0/514664393 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa9500ff140 con 0x7fa9380a3cc0 2024-04-25T13:19:16.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.587+0000 7fa94dffb700 1 -- 172.21.15.31:0/514664393 <== mgr.14148 v2:172.21.15.31:6800/958678691 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa9500ff140 con 0x7fa9380a3cc0 2024-04-25T13:19:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T13:19:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T13:19:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:smithi066 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53430091400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:19:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:smithi136 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD514100122P0SGN 1863G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.589+0000 7fa957102700 1 -- 172.21.15.31:0/514664393 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa9380a3cc0 msgr2=0x7fa9380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.589+0000 7fa957102700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa9380a3cc0 0x7fa9380a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa94000b580 tx=0x7fa940005fb0 comp rx=0 tx=0).stop 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.589+0000 7fa957102700 1 -- 172.21.15.31:0/514664393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa950103560 msgr2=0x7fa9501a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.589+0000 7fa957102700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa950103560 0x7fa9501a23e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa94800b9a0 tx=0x7fa94800bd60 comp rx=0 tx=0).stop 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.589+0000 7fa957102700 1 -- 172.21.15.31:0/514664393 shutdown_connections 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.589+0000 7fa957102700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa9380a3cc0 0x7fa9380a6180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.589+0000 7fa957102700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9501019c0 0x7fa95019aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.589+0000 7fa957102700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa950103560 0x7fa9501a23e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.589+0000 7fa957102700 1 --2- 172.21.15.31:0/514664393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa950102bc0 0x7fa95019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.589+0000 7fa957102700 1 -- 172.21.15.31:0/514664393 >> 172.21.15.31:0/514664393 conn(0x7fa9500fcf60 msgr2=0x7fa9500fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.590+0000 7fa957102700 1 -- 172.21.15.31:0/514664393 shutdown_connections 2024-04-25T13:19:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:16.590+0000 7fa957102700 1 -- 172.21.15.31:0/514664393 wait complete. 2024-04-25T13:19:17.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:16 smithi136 ceph-mon[33038]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T13:19:17.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:16 smithi031 ceph-mon[29457]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T13:19:17.287 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:19:17.297 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T13:19:17.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T13:19:17.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:16 smithi066 ceph-mon[33137]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T13:19:17.549 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:19:18.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:17 smithi136 ceph-mon[33038]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:18.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:17 smithi031 ceph-mon[29457]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:18.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:17 smithi066 ceph-mon[33137]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:18.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:18 smithi031 ceph-mon[29457]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.171+0000 7fddd04bc700 1 -- 172.21.15.31:0/1584639350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8102ba0 msgr2=0x7fddc8103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.171+0000 7fddd04bc700 1 --2- 172.21.15.31:0/1584639350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8102ba0 0x7fddc8103000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fddb80099c0 tx=0x7fddb8009cd0 comp rx=0 tx=0).stop 2024-04-25T13:19:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.171+0000 7fddd04bc700 1 -- 172.21.15.31:0/1584639350 shutdown_connections 2024-04-25T13:19:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.171+0000 7fddd04bc700 1 --2- 172.21.15.31:0/1584639350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc81019a0 0x7fddc8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.171+0000 7fddd04bc700 1 --2- 172.21.15.31:0/1584639350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddc8103540 0x7fddc8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.171+0000 7fddd04bc700 1 --2- 172.21.15.31:0/1584639350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8102ba0 0x7fddc8103000 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.171+0000 7fddd04bc700 1 -- 172.21.15.31:0/1584639350 >> 172.21.15.31:0/1584639350 conn(0x7fddc80fcf40 msgr2=0x7fddc80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.171+0000 7fddd04bc700 1 -- 172.21.15.31:0/1584639350 shutdown_connections 2024-04-25T13:19:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.171+0000 7fddd04bc700 1 -- 172.21.15.31:0/1584639350 wait complete. 2024-04-25T13:19:19.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddd04bc700 1 Processor -- start 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddd04bc700 1 -- start start 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddd04bc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddc81019a0 0x7fddc8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddd04bc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc8102ba0 0x7fddc81991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddd04bc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8103540 0x7fddc81a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddd04bc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fddc810d430 con 0x7fddc8103540 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddd04bc700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fddc810d2b0 con 0x7fddc81019a0 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddd04bc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fddc810d5b0 con 0x7fddc8102ba0 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddcea59700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8103540 0x7fddc81a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddcea59700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8103540 0x7fddc81a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45952/0 (socket says 172.21.15.31:45952) 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddcea59700 1 -- 172.21.15.31:0/919448227 learned_addr learned my addr 172.21.15.31:0/919448227 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddcda57700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc8102ba0 0x7fddc81991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddce258700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddc81019a0 0x7fddc8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddcea59700 1 -- 172.21.15.31:0/919448227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc8102ba0 msgr2=0x7fddc81991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:19.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddcea59700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc8102ba0 0x7fddc81991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddcea59700 1 -- 172.21.15.31:0/919448227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddc81019a0 msgr2=0x7fddc8198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddcea59700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddc81019a0 0x7fddc8198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.172+0000 7fddcea59700 1 -- 172.21.15.31:0/919448227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc81a0890 con 0x7fddc8103540 2024-04-25T13:19:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.173+0000 7fddcea59700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8103540 0x7fddc81a01c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fddc0007ee0 tx=0x7fddc000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.173+0000 7fddbf7fe700 1 -- 172.21.15.31:0/919448227 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0004bb0 con 0x7fddc8103540 2024-04-25T13:19:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.173+0000 7fddd04bc700 1 -- 172.21.15.31:0/919448227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc81a0b80 con 0x7fddc8103540 2024-04-25T13:19:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.173+0000 7fddbf7fe700 1 -- 172.21.15.31:0/919448227 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc0004d50 con 0x7fddc8103540 2024-04-25T13:19:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.173+0000 7fddbf7fe700 1 -- 172.21.15.31:0/919448227 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0005920 con 0x7fddc8103540 2024-04-25T13:19:19.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.173+0000 7fddd04bc700 1 -- 172.21.15.31:0/919448227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc81a1110 con 0x7fddc8103540 2024-04-25T13:19:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.174+0000 7fddd04bc700 1 -- 172.21.15.31:0/919448227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc8069730 con 0x7fddc8103540 2024-04-25T13:19:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.175+0000 7fddbf7fe700 1 -- 172.21.15.31:0/919448227 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fddc001a020 con 0x7fddc8103540 2024-04-25T13:19:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.176+0000 7fddbf7fe700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fddb40a3c40 0x7fddb40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.176+0000 7fddce258700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fddb40a3c40 0x7fddb40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.176+0000 7fddbf7fe700 1 -- 172.21.15.31:0/919448227 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fddc00d0120 con 0x7fddc8103540 2024-04-25T13:19:19.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.176+0000 7fddce258700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fddb40a3c40 0x7fddb40a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fddc819a120 tx=0x7fddc400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:19.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.180+0000 7fddbf7fe700 1 -- 172.21.15.31:0/919448227 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fddc0010070 con 0x7fddc8103540 2024-04-25T13:19:19.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:18 smithi136 ceph-mon[33038]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:19.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:18 smithi066 ceph-mon[33137]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:19.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.438+0000 7fddd04bc700 1 -- 172.21.15.31:0/919448227 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fddc81a14f0 con 0x7fddc8103540 2024-04-25T13:19:19.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.438+0000 7fddbf7fe700 1 -- 172.21.15.31:0/919448227 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+3002 (secure 0 0 0) 0x7fddc009cae0 con 0x7fddc8103540 2024-04-25T13:19:19.438 INFO:teuthology.orchestra.run.smithi031.stdout:e20 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 20 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-25T13:17:40.162672+0000 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-25T13:17:44.194940+0000 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 300 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-25T13:19:19.439 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24511} 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-25T13:19:19.440 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 24511 members: 24511 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.i{0:24511} state up:active seq 5 addr [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b{0:24497} state up:standby-replay seq 1 addr [v2:172.21.15.66:6838/2682191586,v1:172.21.15.66:6839/2682191586] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.31:6834/2096804381,v1:172.21.15.31:6835/2096804381] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.31:6836/1972203542,v1:172.21.15.31:6837/1972203542] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.g{-1:14625} state up:standby seq 1 addr [v2:172.21.15.31:6838/2599371925,v1:172.21.15.31:6839/2599371925] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a{-1:14637} state up:standby seq 1 addr [v2:172.21.15.31:6840/164386980,v1:172.21.15.31:6841/164386980] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.441 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.k{-1:24407} state up:standby seq 1 addr [v2:172.21.15.66:6832/650781414,v1:172.21.15.66:6833/650781414] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.442 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.l{-1:24433} state up:standby seq 1 addr [v2:172.21.15.136:6832/2316613823,v1:172.21.15.136:6833/2316613823] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.442 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.e{-1:24437} state up:standby seq 1 addr [v2:172.21.15.66:6834/2836275175,v1:172.21.15.66:6835/2836275175] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.442 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.h{-1:24464} state up:standby seq 1 addr [v2:172.21.15.66:6836/23034509,v1:172.21.15.66:6837/23034509] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.442 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.f{-1:24478} state up:standby seq 1 addr [v2:172.21.15.136:6834/3086187130,v1:172.21.15.136:6835/3086187130] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.442 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c{-1:24490} state up:standby seq 1 addr [v2:172.21.15.136:6836/3062184830,v1:172.21.15.136:6837/3062184830] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 -- 172.21.15.31:0/919448227 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fddb40a3c40 msgr2=0x7fddb40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fddb40a3c40 0x7fddb40a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fddc819a120 tx=0x7fddc400b410 comp rx=0 tx=0).stop 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 -- 172.21.15.31:0/919448227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8103540 msgr2=0x7fddc81a01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8103540 0x7fddc81a01c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fddc0007ee0 tx=0x7fddc000d3b0 comp rx=0 tx=0).stop 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 -- 172.21.15.31:0/919448227 shutdown_connections 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc8103540 0x7fddc81a01c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fddc81019a0 0x7fddc8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fddb40a3c40 0x7fddb40a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 --2- 172.21.15.31:0/919448227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddc8102ba0 0x7fddc81991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 -- 172.21.15.31:0/919448227 >> 172.21.15.31:0/919448227 conn(0x7fddc80fcf40 msgr2=0x7fddc806a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 -- 172.21.15.31:0/919448227 shutdown_connections 2024-04-25T13:19:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:19.442+0000 7fddd04bc700 1 -- 172.21.15.31:0/919448227 wait complete. 2024-04-25T13:19:19.444 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-04-25T13:19:20.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:19 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/919448227' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:19:20.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T13:19:20.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:19 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/919448227' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:19:20.283 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:19:20.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:19 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/919448227' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:19:20.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:20 smithi031 ceph-mon[29457]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:19:21.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:20 smithi136 ceph-mon[33038]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:19:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:20 smithi066 ceph-mon[33137]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:19:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.071+0000 7ff189939700 1 -- 172.21.15.31:0/414191799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff184103530 msgr2=0x7ff184105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.071+0000 7ff189939700 1 --2- 172.21.15.31:0/414191799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff184103530 0x7ff184105920 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009990 tx=0x7ff17c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:19:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.071+0000 7ff189939700 1 -- 172.21.15.31:0/414191799 shutdown_connections 2024-04-25T13:19:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.071+0000 7ff189939700 1 --2- 172.21.15.31:0/414191799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff184108790 0x7ff18410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.071+0000 7ff189939700 1 --2- 172.21.15.31:0/414191799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff184103530 0x7ff184105920 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:22.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.071+0000 7ff189939700 1 --2- 172.21.15.31:0/414191799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff184105e60 0x7ff184108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:22.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.071+0000 7ff189939700 1 -- 172.21.15.31:0/414191799 >> 172.21.15.31:0/414191799 conn(0x7ff1840fcf40 msgr2=0x7ff1840ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:22.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.071+0000 7ff189939700 1 -- 172.21.15.31:0/414191799 shutdown_connections 2024-04-25T13:19:22.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.071+0000 7ff189939700 1 -- 172.21.15.31:0/414191799 wait complete. 2024-04-25T13:19:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff189939700 1 Processor -- start 2024-04-25T13:19:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff189939700 1 -- start start 2024-04-25T13:19:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff189939700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff184103530 0x7ff184198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:22.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff189939700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff184105e60 0x7ff184199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff189939700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff184108790 0x7ff1841a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff189939700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff18410e550 con 0x7ff184103530 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff189939700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff18410e3d0 con 0x7ff184105e60 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff189939700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff18410e6d0 con 0x7ff184108790 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff182ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff184103530 0x7ff184198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff182ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff184103530 0x7ff184198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34534/0 (socket says 172.21.15.31:34534) 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff182ffd700 1 -- 172.21.15.31:0/958791326 learned_addr learned my addr 172.21.15.31:0/958791326 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff1827fc700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff184105e60 0x7ff184199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff1837fe700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff184108790 0x7ff1841a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff182ffd700 1 -- 172.21.15.31:0/958791326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff184108790 msgr2=0x7ff1841a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff182ffd700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff184108790 0x7ff1841a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.072+0000 7ff182ffd700 1 -- 172.21.15.31:0/958791326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff184105e60 msgr2=0x7ff184199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.073+0000 7ff182ffd700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff184105e60 0x7ff184199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:22.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.073+0000 7ff182ffd700 1 -- 172.21.15.31:0/958791326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff184101320 con 0x7ff184103530 2024-04-25T13:19:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.073+0000 7ff182ffd700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff184103530 0x7ff184198c20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff17c006010 tx=0x7ff17c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.073+0000 7ff188937700 1 -- 172.21.15.31:0/958791326 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c027070 con 0x7ff184103530 2024-04-25T13:19:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.073+0000 7ff189939700 1 -- 172.21.15.31:0/958791326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1841015b0 con 0x7ff184103530 2024-04-25T13:19:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.073+0000 7ff188937700 1 -- 172.21.15.31:0/958791326 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff17c004000 con 0x7ff184103530 2024-04-25T13:19:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.073+0000 7ff188937700 1 -- 172.21.15.31:0/958791326 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c022ba0 con 0x7ff184103530 2024-04-25T13:19:22.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.073+0000 7ff189939700 1 -- 172.21.15.31:0/958791326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff184101a90 con 0x7ff184103530 2024-04-25T13:19:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.074+0000 7ff189939700 1 -- 172.21.15.31:0/958791326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff184069730 con 0x7ff184103530 2024-04-25T13:19:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.075+0000 7ff188937700 1 -- 172.21.15.31:0/958791326 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff17c02a070 con 0x7ff184103530 2024-04-25T13:19:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.076+0000 7ff188937700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1700a3c40 0x7ff1700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.076+0000 7ff1827fc700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1700a3c40 0x7ff1700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.076+0000 7ff188937700 1 -- 172.21.15.31:0/958791326 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7ff17c0d1190 con 0x7ff184103530 2024-04-25T13:19:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.076+0000 7ff1827fc700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1700a3c40 0x7ff1700a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff16c005950 tx=0x7ff16c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.080+0000 7ff188937700 1 -- 172.21.15.31:0/958791326 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff17c013070 con 0x7ff184103530 2024-04-25T13:19:22.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.287+0000 7ff189939700 1 -- 172.21.15.31:0/958791326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7ff1840fe4a0 con 0x7ff184103530 2024-04-25T13:19:22.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.288+0000 7ff188937700 1 -- 172.21.15.31:0/958791326 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5319 (secure 0 0 0) 0x7ff17c0041b0 con 0x7ff184103530 2024-04-25T13:19:22.288 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 90 2024-04-25T13:19:22.288 INFO:teuthology.orchestra.run.smithi031.stdout:fsid e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:19:22.288 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-25T13:09:34.490481+0000 2024-04-25T13:19:22.288 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-25T13:18:32.938284+0000 2024-04-25T13:19:22.288 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T13:19:22.288 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 26 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client luminous 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release squid 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 12 2024-04-25T13:19:22.289 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6802/2594221670,v1:172.21.15.31:6803/2594221670] [v2:172.21.15.31:6804/2594221670,v1:172.21.15.31:6805/2594221670] exists,up 5630157c-602a-4837-a0e0-026f57ea83b0 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6800/786007860,v1:172.21.15.66:6801/786007860] [v2:172.21.15.66:6802/786007860,v1:172.21.15.66:6803/786007860] exists,up f5b3a9a6-6981-4000-a44d-f4ce49325ed6 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/3445625269,v1:172.21.15.136:6801/3445625269] [v2:172.21.15.136:6802/3445625269,v1:172.21.15.136:6803/3445625269] exists,up d3b167a2-c38c-4b5d-a02c-b155ef73624b 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 27 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6810/1987517958,v1:172.21.15.31:6811/1987517958] [v2:172.21.15.31:6812/1987517958,v1:172.21.15.31:6813/1987517958] exists,up 9dfa360c-0171-4df2-ad03-690209f41d77 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 35 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6808/2164191233,v1:172.21.15.66:6809/2164191233] [v2:172.21.15.66:6810/2164191233,v1:172.21.15.66:6811/2164191233] exists,up 4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 40 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/3732026911,v1:172.21.15.136:6809/3732026911] [v2:172.21.15.136:6810/3732026911,v1:172.21.15.136:6811/3732026911] exists,up e70ec561-45b2-4678-9d4a-f94c95a3a29a 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 48 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6818/1320917141,v1:172.21.15.31:6819/1320917141] [v2:172.21.15.31:6820/1320917141,v1:172.21.15.31:6821/1320917141] exists,up 00c2e910-8c20-44b2-ab6d-2eb18f8d8add 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6816/2705866945,v1:172.21.15.66:6817/2705866945] [v2:172.21.15.66:6818/2705866945,v1:172.21.15.66:6819/2705866945] exists,up 50858634-4ead-4b51-9ed1-e5dfe3fd2ceb 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 up in weight 1 up_from 61 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/2823786141,v1:172.21.15.136:6817/2823786141] [v2:172.21.15.136:6818/2823786141,v1:172.21.15.136:6819/2823786141] exists,up 3649fef3-e745-4d64-9601-f09db9203458 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 up in weight 1 up_from 69 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6826/301987340,v1:172.21.15.31:6827/301987340] [v2:172.21.15.31:6828/301987340,v1:172.21.15.31:6829/301987340] exists,up c8974813-c7d4-4aa5-b86f-7c2403d2b29c 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 up in weight 1 up_from 75 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6824/2563017636,v1:172.21.15.66:6825/2563017636] [v2:172.21.15.66:6826/2563017636,v1:172.21.15.66:6827/2563017636] exists,up 14b500a9-fd99-46fd-8405-c10168928b31 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 up in weight 1 up_from 83 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/2269044941,v1:172.21.15.136:6825/2269044941] [v2:172.21.15.136:6826/2269044941,v1:172.21.15.136:6827/2269044941] exists,up fc059f09-0557-40a5-a573-5c304b9bdc25 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.0 [4,6] 2024-04-25T13:19:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.1 [4,9] 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.6 [3,7] 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.e [4,8] 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.11 [5,9] 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.1a [5,7] 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.1b [4,8] 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.25 [5,9] 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.2a [3,6] 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.2e [1,8] 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/1933333469 expires 2024-04-26T13:10:31.702648+0000 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/3047311576 expires 2024-04-26T13:10:31.702648+0000 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/675392433 expires 2024-04-26T13:10:03.802027+0000 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/1458774477 expires 2024-04-26T13:10:03.802027+0000 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/1917834702 expires 2024-04-26T13:10:31.702648+0000 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/1458774477 expires 2024-04-26T13:10:03.802027+0000 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/3193379806 expires 2024-04-26T13:10:03.802027+0000 2024-04-25T13:19:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/3047311576 expires 2024-04-26T13:10:31.702648+0000 2024-04-25T13:19:22.292 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/709772735 expires 2024-04-26T13:10:03.802027+0000 2024-04-25T13:19:22.292 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/179341574 expires 2024-04-26T13:10:31.702648+0000 2024-04-25T13:19:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 -- 172.21.15.31:0/958791326 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1700a3c40 msgr2=0x7ff1700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1700a3c40 0x7ff1700a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff16c005950 tx=0x7ff16c00b410 comp rx=0 tx=0).stop 2024-04-25T13:19:22.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 -- 172.21.15.31:0/958791326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff184103530 msgr2=0x7ff184198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff184103530 0x7ff184198c20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff17c006010 tx=0x7ff17c0057a0 comp rx=0 tx=0).stop 2024-04-25T13:19:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 -- 172.21.15.31:0/958791326 shutdown_connections 2024-04-25T13:19:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff1700a3c40 0x7ff1700a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff184108790 0x7ff1841a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff184103530 0x7ff184198c20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 --2- 172.21.15.31:0/958791326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff184105e60 0x7ff184199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 -- 172.21.15.31:0/958791326 >> 172.21.15.31:0/958791326 conn(0x7ff1840fcf40 msgr2=0x7ff184103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 -- 172.21.15.31:0/958791326 shutdown_connections 2024-04-25T13:19:22.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:22.291+0000 7ff189939700 1 -- 172.21.15.31:0/958791326 wait complete. 2024-04-25T13:19:22.918 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T13:19:22.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:19:23.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 -- 172.21.15.31:0/1003338834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638103b70 msgr2=0x7f163810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 --2- 172.21.15.31:0/1003338834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638103b70 0x7f163810a400 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1630009ca0 tx=0x7f163002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 -- 172.21.15.31:0/1003338834 shutdown_connections 2024-04-25T13:19:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 --2- 172.21.15.31:0/1003338834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638103b70 0x7f163810a400 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 --2- 172.21.15.31:0/1003338834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16381031b0 0x7f1638103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 --2- 172.21.15.31:0/1003338834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1638101fc0 0x7f16381023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 -- 172.21.15.31:0/1003338834 >> 172.21.15.31:0/1003338834 conn(0x7f16380fd550 msgr2=0x7f16380ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 -- 172.21.15.31:0/1003338834 shutdown_connections 2024-04-25T13:19:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 -- 172.21.15.31:0/1003338834 wait complete. 2024-04-25T13:19:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 Processor -- start 2024-04-25T13:19:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 -- start start 2024-04-25T13:19:23.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1638101fc0 0x7f1638198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16381031b0 0x7f16381986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.036+0000 7f163f9cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638103b70 0x7f163819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163f9cd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f163810db00 con 0x7f1638103b70 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163f9cd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f163810d980 con 0x7f1638101fc0 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163f9cd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f163810dc80 con 0x7f16381031b0 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163df6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638103b70 0x7f163819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163df6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638103b70 0x7f163819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34560/0 (socket says 172.21.15.31:34560) 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163df6a640 1 -- 172.21.15.31:0/686734612 learned_addr learned my addr 172.21.15.31:0/686734612 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163cf68640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16381031b0 0x7f16381986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163d769640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1638101fc0 0x7f1638198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163df6a640 1 -- 172.21.15.31:0/686734612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16381031b0 msgr2=0x7f16381986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163df6a640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16381031b0 0x7f16381986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163df6a640 1 -- 172.21.15.31:0/686734612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1638101fc0 msgr2=0x7f1638198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163df6a640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1638101fc0 0x7f1638198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163df6a640 1 -- 172.21.15.31:0/686734612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163819fd90 con 0x7f1638103b70 2024-04-25T13:19:23.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163df6a640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638103b70 0x7f163819f720 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1630031a20 tx=0x7f1630003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:23.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f162e7fc640 1 -- 172.21.15.31:0/686734612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630004240 con 0x7f1638103b70 2024-04-25T13:19:23.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163f9cd640 1 -- 172.21.15.31:0/686734612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16381a0020 con 0x7f1638103b70 2024-04-25T13:19:23.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f162e7fc640 1 -- 172.21.15.31:0/686734612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1630003e10 con 0x7f1638103b70 2024-04-25T13:19:23.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f162e7fc640 1 -- 172.21.15.31:0/686734612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300305c0 con 0x7f1638103b70 2024-04-25T13:19:23.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.037+0000 7f163f9cd640 1 -- 172.21.15.31:0/686734612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16381a0570 con 0x7f1638103b70 2024-04-25T13:19:23.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.038+0000 7f163f9cd640 1 -- 172.21.15.31:0/686734612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1638109380 con 0x7f1638103b70 2024-04-25T13:19:23.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.039+0000 7f162e7fc640 1 -- 172.21.15.31:0/686734612 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f163002f660 con 0x7f1638103b70 2024-04-25T13:19:23.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.040+0000 7f162e7fc640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f16240a3bb0 0x7f16240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.040+0000 7f163d769640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f16240a3bb0 0x7f16240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.040+0000 7f162e7fc640 1 -- 172.21.15.31:0/686734612 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f16300f1f40 con 0x7f1638103b70 2024-04-25T13:19:23.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.041+0000 7f163d769640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f16240a3bb0 0x7f16240a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f16340058c0 tx=0x7f1634005c80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:23.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.044+0000 7f162e7fc640 1 -- 172.21.15.31:0/686734612 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16300be8a0 con 0x7f1638103b70 2024-04-25T13:19:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:22 smithi136 ceph-mon[33038]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:22 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/958791326' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:19:23.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.231+0000 7f163f9cd640 1 -- 172.21.15.31:0/686734612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1638062b50 con 0x7f1638103b70 2024-04-25T13:19:23.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.233+0000 7f162e7fc640 1 -- 172.21.15.31:0/686734612 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22650 (secure 0 0 0) 0x7f16300be1a0 con 0x7f1638103b70 2024-04-25T13:19:23.233 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:23.233 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:18:32.938284+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.2e","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:19:23.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:22 smithi031 ceph-mon[29457]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:23.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:22 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/958791326' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:19:23.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 -- 172.21.15.31:0/686734612 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f16240a3bb0 msgr2=0x7f16240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f16240a3bb0 0x7f16240a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f16340058c0 tx=0x7f1634005c80 comp rx=0 tx=0).stop 2024-04-25T13:19:23.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 -- 172.21.15.31:0/686734612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638103b70 msgr2=0x7f163819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638103b70 0x7f163819f720 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1630031a20 tx=0x7f1630003c20 comp rx=0 tx=0).stop 2024-04-25T13:19:23.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 -- 172.21.15.31:0/686734612 shutdown_connections 2024-04-25T13:19:23.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f16240a3bb0 0x7f16240a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638103b70 0x7f163819f720 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16381031b0 0x7f16381986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 --2- 172.21.15.31:0/686734612 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1638101fc0 0x7f1638198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 -- 172.21.15.31:0/686734612 >> 172.21.15.31:0/686734612 conn(0x7f16380fd550 msgr2=0x7f1638106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:23.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 -- 172.21.15.31:0/686734612 shutdown_connections 2024-04-25T13:19:23.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.235+0000 7f163f9cd640 1 -- 172.21.15.31:0/686734612 wait complete. 2024-04-25T13:19:23.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:13:01.793580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:17:37.994125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:17:39.116456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:19:23.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:19:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.351+0000 7f239bd43640 1 -- 172.21.15.31:0/1341505568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2394107c70 msgr2=0x7f239410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 --2- 172.21.15.31:0/1341505568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2394107c70 0x7f239410a0c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f238c009900 tx=0x7f238c02a410 comp rx=0 tx=0).stop 2024-04-25T13:19:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 -- 172.21.15.31:0/1341505568 shutdown_connections 2024-04-25T13:19:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 --2- 172.21.15.31:0/1341505568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2394107c70 0x7f239410a0c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 --2- 172.21.15.31:0/1341505568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2394105340 0x7f2394107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 --2- 172.21.15.31:0/1341505568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2394102a10 0x7f2394104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 -- 172.21.15.31:0/1341505568 >> 172.21.15.31:0/1341505568 conn(0x7f23940fc400 msgr2=0x7f23940fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 -- 172.21.15.31:0/1341505568 shutdown_connections 2024-04-25T13:19:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 -- 172.21.15.31:0/1341505568 wait complete. 2024-04-25T13:19:23.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 Processor -- start 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 -- start start 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2394102a10 0x7f23941981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2394105340 0x7f23941986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2394107c70 0x7f239419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f239410dae0 con 0x7f2394107c70 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f239410d960 con 0x7f2394102a10 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.352+0000 7f239bd43640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f239410dc60 con 0x7f2394105340 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f239a2e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2394107c70 0x7f239419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f239a2e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2394107c70 0x7f239419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34568/0 (socket says 172.21.15.31:34568) 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f239a2e0640 1 -- 172.21.15.31:0/337250388 learned_addr learned my addr 172.21.15.31:0/337250388 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f23992de640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2394105340 0x7f23941986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f2399adf640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2394102a10 0x7f23941981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f239a2e0640 1 -- 172.21.15.31:0/337250388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2394105340 msgr2=0x7f23941986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f239a2e0640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2394105340 0x7f23941986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f239a2e0640 1 -- 172.21.15.31:0/337250388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2394102a10 msgr2=0x7f23941981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f239a2e0640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2394102a10 0x7f23941981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f239a2e0640 1 -- 172.21.15.31:0/337250388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239419fdd0 con 0x7f2394107c70 2024-04-25T13:19:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f239a2e0640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2394107c70 0x7f239419f760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f238c031500 tx=0x7f238c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f238affd640 1 -- 172.21.15.31:0/337250388 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c03f030 con 0x7f2394107c70 2024-04-25T13:19:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f238affd640 1 -- 172.21.15.31:0/337250388 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238c003aa0 con 0x7f2394107c70 2024-04-25T13:19:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f239bd43640 1 -- 172.21.15.31:0/337250388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23941a0060 con 0x7f2394107c70 2024-04-25T13:19:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f238affd640 1 -- 172.21.15.31:0/337250388 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c035070 con 0x7f2394107c70 2024-04-25T13:19:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.353+0000 7f239bd43640 1 -- 172.21.15.31:0/337250388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23941a0550 con 0x7f2394107c70 2024-04-25T13:19:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.354+0000 7f239bd43640 1 -- 172.21.15.31:0/337250388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23940fdef0 con 0x7f2394107c70 2024-04-25T13:19:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.355+0000 7f238affd640 1 -- 172.21.15.31:0/337250388 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f238c042030 con 0x7f2394107c70 2024-04-25T13:19:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.356+0000 7f238affd640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23800a3bb0 0x7f23800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.356+0000 7f2399adf640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23800a3bb0 0x7f23800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.356+0000 7f238affd640 1 -- 172.21.15.31:0/337250388 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f238c0f10c0 con 0x7f2394107c70 2024-04-25T13:19:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.356+0000 7f2399adf640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23800a3bb0 0x7f23800a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2390005c40 tx=0x7f2390008230 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:23.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.360+0000 7f238affd640 1 -- 172.21.15.31:0/337250388 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f238c0bda80 con 0x7f2394107c70 2024-04-25T13:19:23.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:22 smithi066 ceph-mon[33137]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:23.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:22 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/958791326' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:19:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.545+0000 7f239bd43640 1 -- 172.21.15.31:0/337250388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f239402d5c0 con 0x7f2394107c70 2024-04-25T13:19:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.545+0000 7f238affd640 1 -- 172.21.15.31:0/337250388 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f238c0bd380 con 0x7f2394107c70 2024-04-25T13:19:23.545 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T13:19:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 -- 172.21.15.31:0/337250388 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23800a3bb0 msgr2=0x7f23800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23800a3bb0 0x7f23800a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2390005c40 tx=0x7f2390008230 comp rx=0 tx=0).stop 2024-04-25T13:19:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 -- 172.21.15.31:0/337250388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2394107c70 msgr2=0x7f239419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2394107c70 0x7f239419f760 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f238c031500 tx=0x7f238c031a50 comp rx=0 tx=0).stop 2024-04-25T13:19:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 -- 172.21.15.31:0/337250388 shutdown_connections 2024-04-25T13:19:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f23800a3bb0 0x7f23800a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2394107c70 0x7f239419f760 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2394105340 0x7f23941986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 --2- 172.21.15.31:0/337250388 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2394102a10 0x7f23941981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 -- 172.21.15.31:0/337250388 >> 172.21.15.31:0/337250388 conn(0x7f23940fc400 msgr2=0x7f23941032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 -- 172.21.15.31:0/337250388 shutdown_connections 2024-04-25T13:19:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.547+0000 7f239bd43640 1 -- 172.21.15.31:0/337250388 wait complete. 2024-04-25T13:19:23.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:19:23.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.663+0000 7f0a697fe640 1 -- 172.21.15.31:0/3231902755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a64103b30 msgr2=0x7f0a6410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.663+0000 7f0a697fe640 1 --2- 172.21.15.31:0/3231902755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a64103b30 0x7f0a6410a3c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009ca0 tx=0x7f0a5c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.663+0000 7f0a697fe640 1 -- 172.21.15.31:0/3231902755 shutdown_connections 2024-04-25T13:19:23.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.663+0000 7f0a697fe640 1 --2- 172.21.15.31:0/3231902755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a64103b30 0x7f0a6410a3c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.663+0000 7f0a697fe640 1 --2- 172.21.15.31:0/3231902755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a64103170 0x7f0a641035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.663+0000 7f0a697fe640 1 --2- 172.21.15.31:0/3231902755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a64101f80 0x7f0a641023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.663+0000 7f0a697fe640 1 -- 172.21.15.31:0/3231902755 >> 172.21.15.31:0/3231902755 conn(0x7f0a640fd510 msgr2=0x7f0a640ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:23.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.663+0000 7f0a697fe640 1 -- 172.21.15.31:0/3231902755 shutdown_connections 2024-04-25T13:19:23.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.663+0000 7f0a697fe640 1 -- 172.21.15.31:0/3231902755 wait complete. 2024-04-25T13:19:23.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a697fe640 1 Processor -- start 2024-04-25T13:19:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a697fe640 1 -- start start 2024-04-25T13:19:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a697fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a64101f80 0x7f0a64075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a697fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a64103170 0x7f0a64074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a697fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a64103b30 0x7f0a64074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a697fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6410dac0 con 0x7f0a64103170 2024-04-25T13:19:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a697fe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6410d940 con 0x7f0a64103b30 2024-04-25T13:19:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a697fe640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6410dc40 con 0x7f0a64101f80 2024-04-25T13:19:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a627fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a64103170 0x7f0a64074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a627fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a64103170 0x7f0a64074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34596/0 (socket says 172.21.15.31:34596) 2024-04-25T13:19:23.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a627fc640 1 -- 172.21.15.31:0/1209756821 learned_addr learned my addr 172.21.15.31:0/1209756821 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a62ffd640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a64101f80 0x7f0a64075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a637fe640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a64103b30 0x7f0a64074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a627fc640 1 -- 172.21.15.31:0/1209756821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a64101f80 msgr2=0x7f0a64075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a627fc640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a64101f80 0x7f0a64075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a627fc640 1 -- 172.21.15.31:0/1209756821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a64103b30 msgr2=0x7f0a64074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a627fc640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a64103b30 0x7f0a64074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a627fc640 1 -- 172.21.15.31:0/1209756821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a64075110 con 0x7f0a64103170 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.664+0000 7f0a627fc640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a64103170 0x7f0a64074340 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800d770 tx=0x7f0a5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.665+0000 7f0a4bfff640 1 -- 172.21.15.31:0/1209756821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58017070 con 0x7f0a64103170 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.665+0000 7f0a4bfff640 1 -- 172.21.15.31:0/1209756821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a580054d0 con 0x7f0a64103170 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.665+0000 7f0a697fe640 1 -- 172.21.15.31:0/1209756821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a641a2240 con 0x7f0a64103170 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.665+0000 7f0a4bfff640 1 -- 172.21.15.31:0/1209756821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58004c30 con 0x7f0a64103170 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.665+0000 7f0a697fe640 1 -- 172.21.15.31:0/1209756821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a641a2790 con 0x7f0a64103170 2024-04-25T13:19:23.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.666+0000 7f0a697fe640 1 -- 172.21.15.31:0/1209756821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a64109320 con 0x7f0a64103170 2024-04-25T13:19:23.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.667+0000 7f0a4bfff640 1 -- 172.21.15.31:0/1209756821 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0a58004e50 con 0x7f0a64103170 2024-04-25T13:19:23.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.667+0000 7f0a4bfff640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0a4c0a3c30 0x7f0a4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.668+0000 7f0a62ffd640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0a4c0a3c30 0x7f0a4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.668+0000 7f0a4bfff640 1 -- 172.21.15.31:0/1209756821 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f0a580cfee0 con 0x7f0a64103170 2024-04-25T13:19:23.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.668+0000 7f0a62ffd640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0a4c0a3c30 0x7f0a4c0a60f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a64102fd0 tx=0x7f0a54005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.672+0000 7f0a4bfff640 1 -- 172.21.15.31:0/1209756821 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a5809c840 con 0x7f0a64103170 2024-04-25T13:19:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.862+0000 7f0a697fe640 1 -- 172.21.15.31:0/1209756821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0a64062b50 con 0x7f0a64103170 2024-04-25T13:19:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.862+0000 7f0a4bfff640 1 -- 172.21.15.31:0/1209756821 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f0a5809c140 con 0x7f0a64103170 2024-04-25T13:19:23.862 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T13:19:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 -- 172.21.15.31:0/1209756821 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0a4c0a3c30 msgr2=0x7f0a4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0a4c0a3c30 0x7f0a4c0a60f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a64102fd0 tx=0x7f0a54005f90 comp rx=0 tx=0).stop 2024-04-25T13:19:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 -- 172.21.15.31:0/1209756821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a64103170 msgr2=0x7f0a64074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a64103170 0x7f0a64074340 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800d770 tx=0x7f0a5800dc30 comp rx=0 tx=0).stop 2024-04-25T13:19:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 -- 172.21.15.31:0/1209756821 shutdown_connections 2024-04-25T13:19:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f0a4c0a3c30 0x7f0a4c0a60f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a64103b30 0x7f0a64074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a64103170 0x7f0a64074340 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 --2- 172.21.15.31:0/1209756821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a64101f80 0x7f0a64075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 -- 172.21.15.31:0/1209756821 >> 172.21.15.31:0/1209756821 conn(0x7f0a640fd510 msgr2=0x7f0a641064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 -- 172.21.15.31:0/1209756821 shutdown_connections 2024-04-25T13:19:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.865+0000 7f0a697fe640 1 -- 172.21.15.31:0/1209756821 wait complete. 2024-04-25T13:19:23.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:19:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.982+0000 7f761a87c640 1 -- 172.21.15.31:0/227999067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7614107c90 msgr2=0x7f761410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.982+0000 7f761a87c640 1 --2- 172.21.15.31:0/227999067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7614107c90 0x7f761410a0e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f760c009ca0 tx=0x7f760c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.982+0000 7f761a87c640 1 -- 172.21.15.31:0/227999067 shutdown_connections 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.982+0000 7f761a87c640 1 --2- 172.21.15.31:0/227999067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7614107c90 0x7f761410a0e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.982+0000 7f761a87c640 1 --2- 172.21.15.31:0/227999067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7614105360 0x7f7614107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.982+0000 7f761a87c640 1 --2- 172.21.15.31:0/227999067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7614102a30 0x7f7614104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.982+0000 7f761a87c640 1 -- 172.21.15.31:0/227999067 >> 172.21.15.31:0/227999067 conn(0x7f76140fc440 msgr2=0x7f76140fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.982+0000 7f761a87c640 1 -- 172.21.15.31:0/227999067 shutdown_connections 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.982+0000 7f761a87c640 1 -- 172.21.15.31:0/227999067 wait complete. 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.982+0000 7f761a87c640 1 Processor -- start 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f761a87c640 1 -- start start 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f761a87c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7614102a30 0x7f7614198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f761a87c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7614105360 0x7f7614198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f761a87c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7614107c90 0x7f761419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f761a87c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f761410dba0 con 0x7f7614107c90 2024-04-25T13:19:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f761a87c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f761410da20 con 0x7f7614105360 2024-04-25T13:19:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f761a87c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f761410dd20 con 0x7f7614102a30 2024-04-25T13:19:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f7618e19640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7614107c90 0x7f761419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f7618e19640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7614107c90 0x7f761419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34610/0 (socket says 172.21.15.31:34610) 2024-04-25T13:19:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f7618e19640 1 -- 172.21.15.31:0/4085002372 learned_addr learned my addr 172.21.15.31:0/4085002372 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f76137fe640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7614105360 0x7f7614198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f7613fff640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7614102a30 0x7f7614198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f7618e19640 1 -- 172.21.15.31:0/4085002372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7614102a30 msgr2=0x7f7614198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f7618e19640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7614102a30 0x7f7614198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f7618e19640 1 -- 172.21.15.31:0/4085002372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7614105360 msgr2=0x7f7614198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f7618e19640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7614105360 0x7f7614198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f7618e19640 1 -- 172.21.15.31:0/4085002372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761419fe30 con 0x7f7614107c90 2024-04-25T13:19:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f7613fff640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7614102a30 0x7f7614198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.983+0000 7f7618e19640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7614107c90 0x7f761419f7c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f760c033040 tx=0x7f760c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.984+0000 7f76117fa640 1 -- 172.21.15.31:0/4085002372 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0317c0 con 0x7f7614107c90 2024-04-25T13:19:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.984+0000 7f76117fa640 1 -- 172.21.15.31:0/4085002372 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760c0044f0 con 0x7f7614107c90 2024-04-25T13:19:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.984+0000 7f76117fa640 1 -- 172.21.15.31:0/4085002372 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c02fd00 con 0x7f7614107c90 2024-04-25T13:19:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.984+0000 7f761a87c640 1 -- 172.21.15.31:0/4085002372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76141a00c0 con 0x7f7614107c90 2024-04-25T13:19:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.984+0000 7f761a87c640 1 -- 172.21.15.31:0/4085002372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76141a0690 con 0x7f7614107c90 2024-04-25T13:19:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.985+0000 7f761a87c640 1 -- 172.21.15.31:0/4085002372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76140fdf30 con 0x7f7614107c90 2024-04-25T13:19:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.986+0000 7f76117fa640 1 -- 172.21.15.31:0/4085002372 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f760c004040 con 0x7f7614107c90 2024-04-25T13:19:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.986+0000 7f76117fa640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f75fc0a3bb0 0x7f75fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.986+0000 7f7613fff640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f75fc0a3bb0 0x7f75fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.987+0000 7f76117fa640 1 -- 172.21.15.31:0/4085002372 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f760c0f1a30 con 0x7f7614107c90 2024-04-25T13:19:23.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.987+0000 7f7613fff640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f75fc0a3bb0 0x7f75fc0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7604009930 tx=0x7f7604008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:23.991+0000 7f76117fa640 1 -- 172.21.15.31:0/4085002372 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f760c0be3f0 con 0x7f7614107c90 2024-04-25T13:19:24.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.179+0000 7f761a87c640 1 -- 172.21.15.31:0/4085002372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7614062b50 con 0x7f7614107c90 2024-04-25T13:19:24.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.180+0000 7f76117fa640 1 -- 172.21.15.31:0/4085002372 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f760c0098b0 con 0x7f7614107c90 2024-04-25T13:19:24.180 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T13:19:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 -- 172.21.15.31:0/4085002372 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f75fc0a3bb0 msgr2=0x7f75fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f75fc0a3bb0 0x7f75fc0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7604009930 tx=0x7f7604008040 comp rx=0 tx=0).stop 2024-04-25T13:19:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 -- 172.21.15.31:0/4085002372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7614107c90 msgr2=0x7f761419f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7614107c90 0x7f761419f7c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f760c033040 tx=0x7f760c003ba0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 -- 172.21.15.31:0/4085002372 shutdown_connections 2024-04-25T13:19:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f75fc0a3bb0 0x7f75fc0a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7614107c90 0x7f761419f7c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7614105360 0x7f7614198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 --2- 172.21.15.31:0/4085002372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7614102a30 0x7f7614198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 -- 172.21.15.31:0/4085002372 >> 172.21.15.31:0/4085002372 conn(0x7f76140fc440 msgr2=0x7f7614103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:24.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 -- 172.21.15.31:0/4085002372 shutdown_connections 2024-04-25T13:19:24.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.182+0000 7f761a87c640 1 -- 172.21.15.31:0/4085002372 wait complete. 2024-04-25T13:19:24.192 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:23 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/686734612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:24.252 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:23 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/337250388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:24.253 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:23 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1209756821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:24.253 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:23 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/686734612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:24.253 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:23 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/337250388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:24.253 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:23 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1209756821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:24.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.356+0000 7fa5e4e64640 1 -- 172.21.15.31:0/2254947705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e0102a00 msgr2=0x7fa5e0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.356+0000 7fa5e4e64640 1 --2- 172.21.15.31:0/2254947705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e0102a00 0x7fa5e0109320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009ca0 tx=0x7fa5d802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.356+0000 7fa5e4e64640 1 -- 172.21.15.31:0/2254947705 shutdown_connections 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.356+0000 7fa5e4e64640 1 --2- 172.21.15.31:0/2254947705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e0102a00 0x7fa5e0109320 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.356+0000 7fa5e4e64640 1 --2- 172.21.15.31:0/2254947705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5e0102040 0x7fa5e01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.356+0000 7fa5e4e64640 1 --2- 172.21.15.31:0/2254947705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5e0100e60 0x7fa5e0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.356+0000 7fa5e4e64640 1 -- 172.21.15.31:0/2254947705 >> 172.21.15.31:0/2254947705 conn(0x7fa5e00fc440 msgr2=0x7fa5e00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.356+0000 7fa5e4e64640 1 -- 172.21.15.31:0/2254947705 shutdown_connections 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.356+0000 7fa5e4e64640 1 -- 172.21.15.31:0/2254947705 wait complete. 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5e4e64640 1 Processor -- start 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5e4e64640 1 -- start start 2024-04-25T13:19:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5e4e64640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5e0100e60 0x7fa5e019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5e4e64640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5e0102040 0x7fa5e019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5e4e64640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e0102a00 0x7fa5e01a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5e4e64640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e010caf0 con 0x7fa5e0102a00 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5e4e64640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e010c970 con 0x7fa5e0102040 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5e4e64640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e010cc70 con 0x7fa5e0100e60 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5ded9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e0102a00 0x7fa5e01a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5ded9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e0102a00 0x7fa5e01a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34618/0 (socket says 172.21.15.31:34618) 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5ded9d640 1 -- 172.21.15.31:0/3512947832 learned_addr learned my addr 172.21.15.31:0/3512947832 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5ddd9b640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5e0102040 0x7fa5e019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5de59c640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5e0100e60 0x7fa5e019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5ded9d640 1 -- 172.21.15.31:0/3512947832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5e0100e60 msgr2=0x7fa5e019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5ded9d640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5e0100e60 0x7fa5e019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5ded9d640 1 -- 172.21.15.31:0/3512947832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5e0102040 msgr2=0x7fa5e019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5ded9d640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5e0102040 0x7fa5e019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5ded9d640 1 -- 172.21.15.31:0/3512947832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e01a2150 con 0x7fa5e0102a00 2024-04-25T13:19:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.357+0000 7fa5de59c640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5e0100e60 0x7fa5e019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.358+0000 7fa5ded9d640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e0102a00 0x7fa5e01a1ae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8031a20 tx=0x7fa5d8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.358+0000 7fa5cf7fe640 1 -- 172.21.15.31:0/3512947832 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8004240 con 0x7fa5e0102a00 2024-04-25T13:19:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.358+0000 7fa5cf7fe640 1 -- 172.21.15.31:0/3512947832 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5d8003e10 con 0x7fa5e0102a00 2024-04-25T13:19:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.358+0000 7fa5e4e64640 1 -- 172.21.15.31:0/3512947832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e01a23e0 con 0x7fa5e0102a00 2024-04-25T13:19:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.358+0000 7fa5cf7fe640 1 -- 172.21.15.31:0/3512947832 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d802fd90 con 0x7fa5e0102a00 2024-04-25T13:19:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.358+0000 7fa5e4e64640 1 -- 172.21.15.31:0/3512947832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e01a29b0 con 0x7fa5e0102a00 2024-04-25T13:19:24.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.359+0000 7fa5e4e64640 1 -- 172.21.15.31:0/3512947832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e01a2c10 con 0x7fa5e0102a00 2024-04-25T13:19:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.360+0000 7fa5cf7fe640 1 -- 172.21.15.31:0/3512947832 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa5d80043e0 con 0x7fa5e0102a00 2024-04-25T13:19:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.360+0000 7fa5cf7fe640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa5c80a3bb0 0x7fa5c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.360+0000 7fa5de59c640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa5c80a3bb0 0x7fa5c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.361+0000 7fa5cf7fe640 1 -- 172.21.15.31:0/3512947832 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fa5d80f1d10 con 0x7fa5e0102a00 2024-04-25T13:19:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.361+0000 7fa5de59c640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa5c80a3bb0 0x7fa5c80a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0101bb0 tx=0x7fa5d0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:24.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.365+0000 7fa5cf7fe640 1 -- 172.21.15.31:0/3512947832 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5d80be6d0 con 0x7fa5e0102a00 2024-04-25T13:19:24.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:23 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/686734612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:24.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:23 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/337250388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:24.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:23 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1209756821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:24.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.593+0000 7fa5e4e64640 1 -- 172.21.15.31:0/3512947832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5e01a2de0 con 0x7fa5e0102a00 2024-04-25T13:19:24.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.593+0000 7fa5cf7fe640 1 -- 172.21.15.31:0/3512947832 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11798 (secure 0 0 0) 0x7fa5d80bdfd0 con 0x7fa5e0102a00 2024-04-25T13:19:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:24.593 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2599371925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2599371925},{"type":"v1","addr":"172.21.15.31:6839","nonce":2599371925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/164386980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":164386980},{"type":"v1","addr":"172.21.15.31:6841","nonce":164386980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:17:40.162672+0000","modified":"2024-04-25T13:17:44.194940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2682191586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2682191586},{"type":"v1","addr":"172.21.15.66:6839","nonce":2682191586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24511]},"id":1}]} 2024-04-25T13:19:24.594 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-04-25T13:19:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 -- 172.21.15.31:0/3512947832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa5c80a3bb0 msgr2=0x7fa5c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa5c80a3bb0 0x7fa5c80a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0101bb0 tx=0x7fa5d0008040 comp rx=0 tx=0).stop 2024-04-25T13:19:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 -- 172.21.15.31:0/3512947832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e0102a00 msgr2=0x7fa5e01a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e0102a00 0x7fa5e01a1ae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8031a20 tx=0x7fa5d8003c20 comp rx=0 tx=0).stop 2024-04-25T13:19:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 -- 172.21.15.31:0/3512947832 shutdown_connections 2024-04-25T13:19:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa5c80a3bb0 0x7fa5c80a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5e0102a00 0x7fa5e01a1ae0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5e0102040 0x7fa5e019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 --2- 172.21.15.31:0/3512947832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5e0100e60 0x7fa5e019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 -- 172.21.15.31:0/3512947832 >> 172.21.15.31:0/3512947832 conn(0x7fa5e00fc440 msgr2=0x7fa5e00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 -- 172.21.15.31:0/3512947832 shutdown_connections 2024-04-25T13:19:24.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.595+0000 7fa5e4e64640 1 -- 172.21.15.31:0/3512947832 wait complete. 2024-04-25T13:19:24.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:19:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.709+0000 7f7407d8e640 1 -- 172.21.15.31:0/3218588600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7400103b70 msgr2=0x7f740010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.709+0000 7f7407d8e640 1 --2- 172.21.15.31:0/3218588600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7400103b70 0x7f740010a400 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009ca0 tx=0x7f73f802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.709+0000 7f7407d8e640 1 -- 172.21.15.31:0/3218588600 shutdown_connections 2024-04-25T13:19:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.709+0000 7f7407d8e640 1 --2- 172.21.15.31:0/3218588600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7400103b70 0x7f740010a400 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.709+0000 7f7407d8e640 1 --2- 172.21.15.31:0/3218588600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74001031b0 0x7f7400103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.709+0000 7f7407d8e640 1 --2- 172.21.15.31:0/3218588600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7400101fc0 0x7f74001023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.709+0000 7f7407d8e640 1 -- 172.21.15.31:0/3218588600 >> 172.21.15.31:0/3218588600 conn(0x7f74000fd550 msgr2=0x7f74000ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.709+0000 7f7407d8e640 1 -- 172.21.15.31:0/3218588600 shutdown_connections 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.709+0000 7f7407d8e640 1 -- 172.21.15.31:0/3218588600 wait complete. 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.709+0000 7f7407d8e640 1 Processor -- start 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.709+0000 7f7407d8e640 1 -- start start 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7407d8e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7400101fc0 0x7f7400198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7407d8e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74001031b0 0x7f74001986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7407d8e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7400103b70 0x7f740019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7407d8e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f740010db00 con 0x7f74001031b0 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7407d8e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f740010d980 con 0x7f7400101fc0 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7407d8e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f740010dc80 con 0x7f7400103b70 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7405329640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74001031b0 0x7f74001986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7405329640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74001031b0 0x7f74001986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34638/0 (socket says 172.21.15.31:34638) 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7405329640 1 -- 172.21.15.31:0/1776019809 learned_addr learned my addr 172.21.15.31:0/1776019809 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7405b2a640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7400101fc0 0x7f7400198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f740632b640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7400103b70 0x7f740019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7405329640 1 -- 172.21.15.31:0/1776019809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7400103b70 msgr2=0x7f740019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7405329640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7400103b70 0x7f740019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7405329640 1 -- 172.21.15.31:0/1776019809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7400101fc0 msgr2=0x7f7400198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7405329640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7400101fc0 0x7f7400198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7405329640 1 -- 172.21.15.31:0/1776019809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740019fdb0 con 0x7f74001031b0 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f740632b640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7400103b70 0x7f740019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7405329640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74001031b0 0x7f74001986d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f73f000d770 tx=0x7f73f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f73f6ffd640 1 -- 172.21.15.31:0/1776019809 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0017070 con 0x7f74001031b0 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f73f6ffd640 1 -- 172.21.15.31:0/1776019809 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73f00054d0 con 0x7f74001031b0 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f7407d8e640 1 -- 172.21.15.31:0/1776019809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74001a00a0 con 0x7f74001031b0 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.710+0000 7f73f6ffd640 1 -- 172.21.15.31:0/1776019809 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0004c30 con 0x7f74001031b0 2024-04-25T13:19:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.711+0000 7f7407d8e640 1 -- 172.21.15.31:0/1776019809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74001a05f0 con 0x7f74001031b0 2024-04-25T13:19:24.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.712+0000 7f7407d8e640 1 -- 172.21.15.31:0/1776019809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74001092e0 con 0x7f74001031b0 2024-04-25T13:19:24.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.713+0000 7f73f6ffd640 1 -- 172.21.15.31:0/1776019809 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f73f0004df0 con 0x7f74001031b0 2024-04-25T13:19:24.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.713+0000 7f73f6ffd640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f73ec0a3c30 0x7f73ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:24.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.713+0000 7f7405b2a640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f73ec0a3c30 0x7f73ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:24.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.713+0000 7f73f6ffd640 1 -- 172.21.15.31:0/1776019809 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f73f00cfe30 con 0x7f74001031b0 2024-04-25T13:19:24.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.714+0000 7f7405b2a640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f73ec0a3c30 0x7f73ec0a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f73fc005fd0 tx=0x7f73fc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:24.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.718+0000 7f73f6ffd640 1 -- 172.21.15.31:0/1776019809 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73f009c7f0 con 0x7f74001031b0 2024-04-25T13:19:24.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.909+0000 7f7407d8e640 1 -- 172.21.15.31:0/1776019809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7400062b50 con 0x7f74001031b0 2024-04-25T13:19:24.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.911+0000 7f73f6ffd640 1 -- 172.21.15.31:0/1776019809 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22650 (secure 0 0 0) 0x7f73f009c0f0 con 0x7f74001031b0 2024-04-25T13:19:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:24.911 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:18:32.938284+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.2e","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:19:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 -- 172.21.15.31:0/1776019809 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f73ec0a3c30 msgr2=0x7f73ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f73ec0a3c30 0x7f73ec0a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f73fc005fd0 tx=0x7f73fc00b410 comp rx=0 tx=0).stop 2024-04-25T13:19:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 -- 172.21.15.31:0/1776019809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74001031b0 msgr2=0x7f74001986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74001031b0 0x7f74001986d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f73f000d770 tx=0x7f73f000dc30 comp rx=0 tx=0).stop 2024-04-25T13:19:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 -- 172.21.15.31:0/1776019809 shutdown_connections 2024-04-25T13:19:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f73ec0a3c30 0x7f73ec0a60f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7400103b70 0x7f740019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74001031b0 0x7f74001986d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 --2- 172.21.15.31:0/1776019809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7400101fc0 0x7f7400198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 -- 172.21.15.31:0/1776019809 >> 172.21.15.31:0/1776019809 conn(0x7f74000fd550 msgr2=0x7f74001064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 -- 172.21.15.31:0/1776019809 shutdown_connections 2024-04-25T13:19:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:24.913+0000 7f7407d8e640 1 -- 172.21.15.31:0/1776019809 wait complete. 2024-04-25T13:19:24.924 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:13:01.793580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:17:37.994125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:17:39.116456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:19:24.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:19:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.030+0000 7ff8453c6640 1 -- 172.21.15.31:0/2395318673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff840102a30 msgr2=0x7ff840104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.030+0000 7ff8453c6640 1 --2- 172.21.15.31:0/2395318673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff840102a30 0x7ff840104e20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff830009900 tx=0x7ff83002a410 comp rx=0 tx=0).stop 2024-04-25T13:19:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.030+0000 7ff8453c6640 1 -- 172.21.15.31:0/2395318673 shutdown_connections 2024-04-25T13:19:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.030+0000 7ff8453c6640 1 --2- 172.21.15.31:0/2395318673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff840107c90 0x7ff84010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.030+0000 7ff8453c6640 1 --2- 172.21.15.31:0/2395318673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff840105360 0x7ff840107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.030+0000 7ff8453c6640 1 --2- 172.21.15.31:0/2395318673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff840102a30 0x7ff840104e20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.030+0000 7ff8453c6640 1 -- 172.21.15.31:0/2395318673 >> 172.21.15.31:0/2395318673 conn(0x7ff8400fc440 msgr2=0x7ff8400fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.030+0000 7ff8453c6640 1 -- 172.21.15.31:0/2395318673 shutdown_connections 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.030+0000 7ff8453c6640 1 -- 172.21.15.31:0/2395318673 wait complete. 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.030+0000 7ff8453c6640 1 Processor -- start 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff8453c6640 1 -- start start 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff8453c6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff840102a30 0x7ff840198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff8453c6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff840105360 0x7ff8401987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff8453c6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff840107c90 0x7ff84019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff8453c6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff84010daf0 con 0x7ff840105360 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff8453c6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff84010d970 con 0x7ff840107c90 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff8453c6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff84010dc70 con 0x7ff840102a30 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff83e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff840105360 0x7ff8401987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff83e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff840105360 0x7ff8401987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34660/0 (socket says 172.21.15.31:34660) 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff83e7fc640 1 -- 172.21.15.31:0/2492656845 learned_addr learned my addr 172.21.15.31:0/2492656845 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff83f7fe640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff840107c90 0x7ff84019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff83effd640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff840102a30 0x7ff840198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff83e7fc640 1 -- 172.21.15.31:0/2492656845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff840102a30 msgr2=0x7ff840198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff83e7fc640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff840102a30 0x7ff840198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff83e7fc640 1 -- 172.21.15.31:0/2492656845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff840107c90 msgr2=0x7ff84019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff83e7fc640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff840107c90 0x7ff84019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff83e7fc640 1 -- 172.21.15.31:0/2492656845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84019fe80 con 0x7ff840105360 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.031+0000 7ff83e7fc640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff840105360 0x7ff8401987a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff83400d740 tx=0x7ff83400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.032+0000 7ff827fff640 1 -- 172.21.15.31:0/2492656845 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834017070 con 0x7ff840105360 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.032+0000 7ff827fff640 1 -- 172.21.15.31:0/2492656845 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff83400de50 con 0x7ff840105360 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.032+0000 7ff827fff640 1 -- 172.21.15.31:0/2492656845 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834004ce0 con 0x7ff840105360 2024-04-25T13:19:25.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.032+0000 7ff8453c6640 1 -- 172.21.15.31:0/2492656845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8401a0170 con 0x7ff840105360 2024-04-25T13:19:25.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.032+0000 7ff8453c6640 1 -- 172.21.15.31:0/2492656845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8401a0670 con 0x7ff840105360 2024-04-25T13:19:25.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.033+0000 7ff8453c6640 1 -- 172.21.15.31:0/2492656845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8400fdf30 con 0x7ff840105360 2024-04-25T13:19:25.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.034+0000 7ff827fff640 1 -- 172.21.15.31:0/2492656845 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff834004e80 con 0x7ff840105360 2024-04-25T13:19:25.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.034+0000 7ff827fff640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff8280a3bb0 0x7ff8280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.034+0000 7ff83effd640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff8280a3bb0 0x7ff8280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.035+0000 7ff827fff640 1 -- 172.21.15.31:0/2492656845 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7ff8340d01e0 con 0x7ff840105360 2024-04-25T13:19:25.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.035+0000 7ff83effd640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff8280a3bb0 0x7ff8280a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff830000c00 tx=0x7ff8300314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:25.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.038+0000 7ff827fff640 1 -- 172.21.15.31:0/2492656845 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff83409cba0 con 0x7ff840105360 2024-04-25T13:19:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:24 smithi136 ceph-mon[33038]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:19:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:24 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/4085002372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:24 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3512947832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:24 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1776019809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.225+0000 7ff8453c6640 1 -- 172.21.15.31:0/2492656845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff84002d810 con 0x7ff840105360 2024-04-25T13:19:25.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.226+0000 7ff827fff640 1 -- 172.21.15.31:0/2492656845 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7ff83409c4a0 con 0x7ff840105360 2024-04-25T13:19:25.226 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 -- 172.21.15.31:0/2492656845 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff8280a3bb0 msgr2=0x7ff8280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff8280a3bb0 0x7ff8280a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff830000c00 tx=0x7ff8300314a0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 -- 172.21.15.31:0/2492656845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff840105360 msgr2=0x7ff8401987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff840105360 0x7ff8401987a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff83400d740 tx=0x7ff83400dc00 comp rx=0 tx=0).stop 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 -- 172.21.15.31:0/2492656845 shutdown_connections 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff8280a3bb0 0x7ff8280a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff840107c90 0x7ff84019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff840105360 0x7ff8401987a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 --2- 172.21.15.31:0/2492656845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff840102a30 0x7ff840198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 -- 172.21.15.31:0/2492656845 >> 172.21.15.31:0/2492656845 conn(0x7ff8400fc440 msgr2=0x7ff840103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 -- 172.21.15.31:0/2492656845 shutdown_connections 2024-04-25T13:19:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.228+0000 7ff8453c6640 1 -- 172.21.15.31:0/2492656845 wait complete. 2024-04-25T13:19:25.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:24 smithi031 ceph-mon[29457]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:19:25.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:24 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/4085002372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:25.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:24 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3512947832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:24 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1776019809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:25.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:19:25.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.346+0000 7fb23c520640 1 -- 172.21.15.31:0/1677398603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb234102980 msgr2=0x7fb2341092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.346+0000 7fb23c520640 1 --2- 172.21.15.31:0/1677398603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb234102980 0x7fb2341092a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009ca0 tx=0x7fb22c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.347+0000 7fb23c520640 1 -- 172.21.15.31:0/1677398603 shutdown_connections 2024-04-25T13:19:25.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.347+0000 7fb23c520640 1 --2- 172.21.15.31:0/1677398603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb234102980 0x7fb2341092a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.347+0000 7fb23c520640 1 --2- 172.21.15.31:0/1677398603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb234101fc0 0x7fb234102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.347+0000 7fb23c520640 1 --2- 172.21.15.31:0/1677398603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb234100e70 0x7fb234101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.347+0000 7fb23c520640 1 -- 172.21.15.31:0/1677398603 >> 172.21.15.31:0/1677398603 conn(0x7fb2340fc3e0 msgr2=0x7fb2340fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:25.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.347+0000 7fb23c520640 1 -- 172.21.15.31:0/1677398603 shutdown_connections 2024-04-25T13:19:25.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.347+0000 7fb23c520640 1 -- 172.21.15.31:0/1677398603 wait complete. 2024-04-25T13:19:25.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.347+0000 7fb23c520640 1 Processor -- start 2024-04-25T13:19:25.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.347+0000 7fb23c520640 1 -- start start 2024-04-25T13:19:25.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23c520640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb234100e70 0x7fb234198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23c520640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb234101fc0 0x7fb2341986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23c520640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb234102980 0x7fb23419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23c520640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb23410ca10 con 0x7fb234102980 2024-04-25T13:19:25.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23c520640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb23410c890 con 0x7fb234101fc0 2024-04-25T13:19:25.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23c520640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb23410cb90 con 0x7fb234100e70 2024-04-25T13:19:25.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23aabd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb234102980 0x7fb23419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23aabd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb234102980 0x7fb23419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34668/0 (socket says 172.21.15.31:34668) 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23aabd640 1 -- 172.21.15.31:0/720870967 learned_addr learned my addr 172.21.15.31:0/720870967 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb239abb640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb234101fc0 0x7fb2341986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23a2bc640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb234100e70 0x7fb234198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23aabd640 1 -- 172.21.15.31:0/720870967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb234100e70 msgr2=0x7fb234198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23aabd640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb234100e70 0x7fb234198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23aabd640 1 -- 172.21.15.31:0/720870967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb234101fc0 msgr2=0x7fb2341986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23aabd640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb234101fc0 0x7fb2341986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23aabd640 1 -- 172.21.15.31:0/720870967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23419fd90 con 0x7fb234102980 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23a2bc640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb234100e70 0x7fb234198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23aabd640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb234102980 0x7fb23419f720 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb22c033040 tx=0x7fb22c0317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb22b7fe640 1 -- 172.21.15.31:0/720870967 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c003b70 con 0x7fb234102980 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb23c520640 1 -- 172.21.15.31:0/720870967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2341a0020 con 0x7fb234102980 2024-04-25T13:19:25.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb22b7fe640 1 -- 172.21.15.31:0/720870967 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c003d10 con 0x7fb234102980 2024-04-25T13:19:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.348+0000 7fb22b7fe640 1 -- 172.21.15.31:0/720870967 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c02fbd0 con 0x7fb234102980 2024-04-25T13:19:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.349+0000 7fb23c520640 1 -- 172.21.15.31:0/720870967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2341a0570 con 0x7fb234102980 2024-04-25T13:19:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.349+0000 7fb23c520640 1 -- 172.21.15.31:0/720870967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2341a07d0 con 0x7fb234102980 2024-04-25T13:19:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.351+0000 7fb22b7fe640 1 -- 172.21.15.31:0/720870967 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb22c02fd70 con 0x7fb234102980 2024-04-25T13:19:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.352+0000 7fb22b7fe640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2200a3bb0 0x7fb2200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.352+0000 7fb23a2bc640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2200a3bb0 0x7fb2200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.352+0000 7fb22b7fe640 1 -- 172.21.15.31:0/720870967 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fb22c0f1c10 con 0x7fb234102980 2024-04-25T13:19:25.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.352+0000 7fb23a2bc640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2200a3bb0 0x7fb2200a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb2240058c0 tx=0x7fb224005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.356+0000 7fb22b7fe640 1 -- 172.21.15.31:0/720870967 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb22c0be570 con 0x7fb234102980 2024-04-25T13:19:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:24 smithi066 ceph-mon[33137]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:19:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:24 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/4085002372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:24 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3512947832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:24 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1776019809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:25.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.541+0000 7fb23c520640 1 -- 172.21.15.31:0/720870967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb2341a09a0 con 0x7fb234102980 2024-04-25T13:19:25.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.542+0000 7fb22b7fe640 1 -- 172.21.15.31:0/720870967 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fb22c0bde70 con 0x7fb234102980 2024-04-25T13:19:25.542 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T13:19:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.543+0000 7fb23c520640 1 -- 172.21.15.31:0/720870967 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2200a3bb0 msgr2=0x7fb2200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.544+0000 7fb23c520640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2200a3bb0 0x7fb2200a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb2240058c0 tx=0x7fb224005d30 comp rx=0 tx=0).stop 2024-04-25T13:19:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.544+0000 7fb23c520640 1 -- 172.21.15.31:0/720870967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb234102980 msgr2=0x7fb23419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.544+0000 7fb23c520640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb234102980 0x7fb23419f720 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb22c033040 tx=0x7fb22c0317c0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.544+0000 7fb23c520640 1 -- 172.21.15.31:0/720870967 shutdown_connections 2024-04-25T13:19:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.544+0000 7fb23c520640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb2200a3bb0 0x7fb2200a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.544+0000 7fb23c520640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb234102980 0x7fb23419f720 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.544+0000 7fb23c520640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb234101fc0 0x7fb2341986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.544+0000 7fb23c520640 1 --2- 172.21.15.31:0/720870967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb234100e70 0x7fb234198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.544+0000 7fb23c520640 1 -- 172.21.15.31:0/720870967 >> 172.21.15.31:0/720870967 conn(0x7fb2340fc3e0 msgr2=0x7fb2340fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.544+0000 7fb23c520640 1 -- 172.21.15.31:0/720870967 shutdown_connections 2024-04-25T13:19:25.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.544+0000 7fb23c520640 1 -- 172.21.15.31:0/720870967 wait complete. 2024-04-25T13:19:25.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:19:25.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.662+0000 7fb5014e8640 1 -- 172.21.15.31:0/4145381725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc102a30 msgr2=0x7fb4fc104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.662+0000 7fb5014e8640 1 --2- 172.21.15.31:0/4145381725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc102a30 0x7fb4fc104e20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009900 tx=0x7fb4f402a410 comp rx=0 tx=0).stop 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.662+0000 7fb5014e8640 1 -- 172.21.15.31:0/4145381725 shutdown_connections 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.662+0000 7fb5014e8640 1 --2- 172.21.15.31:0/4145381725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4fc107c90 0x7fb4fc10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.662+0000 7fb5014e8640 1 --2- 172.21.15.31:0/4145381725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc105360 0x7fb4fc107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.662+0000 7fb5014e8640 1 --2- 172.21.15.31:0/4145381725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc102a30 0x7fb4fc104e20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.662+0000 7fb5014e8640 1 -- 172.21.15.31:0/4145381725 >> 172.21.15.31:0/4145381725 conn(0x7fb4fc0fc440 msgr2=0x7fb4fc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.662+0000 7fb5014e8640 1 -- 172.21.15.31:0/4145381725 shutdown_connections 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.662+0000 7fb5014e8640 1 -- 172.21.15.31:0/4145381725 wait complete. 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb5014e8640 1 Processor -- start 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb5014e8640 1 -- start start 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb5014e8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4fc102a30 0x7fb4fc1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb5014e8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc105360 0x7fb4fc1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb5014e8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc107c90 0x7fb4fc19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb5014e8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc10dba0 con 0x7fb4fc105360 2024-04-25T13:19:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb5014e8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc10da20 con 0x7fb4fc102a30 2024-04-25T13:19:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb5014e8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc10dd20 con 0x7fb4fc107c90 2024-04-25T13:19:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb4fa7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc105360 0x7fb4fc1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb4fa7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc105360 0x7fb4fc1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34684/0 (socket says 172.21.15.31:34684) 2024-04-25T13:19:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb4fa7fc640 1 -- 172.21.15.31:0/3545045340 learned_addr learned my addr 172.21.15.31:0/3545045340 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb4fb7fe640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc107c90 0x7fb4fc19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb4faffd640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4fc102a30 0x7fb4fc1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb4fa7fc640 1 -- 172.21.15.31:0/3545045340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc107c90 msgr2=0x7fb4fc19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb4fa7fc640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc107c90 0x7fb4fc19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb4fa7fc640 1 -- 172.21.15.31:0/3545045340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4fc102a30 msgr2=0x7fb4fc1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb4fa7fc640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4fc102a30 0x7fb4fc1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.663+0000 7fb4fa7fc640 1 -- 172.21.15.31:0/3545045340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4fc19fe30 con 0x7fb4fc105360 2024-04-25T13:19:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.664+0000 7fb4fa7fc640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc105360 0x7fb4fc1987f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400c970 tx=0x7fb4e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.664+0000 7fb4f3fff640 1 -- 172.21.15.31:0/3545045340 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4014070 con 0x7fb4fc105360 2024-04-25T13:19:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.664+0000 7fb5014e8640 1 -- 172.21.15.31:0/3545045340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4fc1a0120 con 0x7fb4fc105360 2024-04-25T13:19:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.664+0000 7fb4f3fff640 1 -- 172.21.15.31:0/3545045340 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e4004440 con 0x7fb4fc105360 2024-04-25T13:19:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.664+0000 7fb4f3fff640 1 -- 172.21.15.31:0/3545045340 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4003e50 con 0x7fb4fc105360 2024-04-25T13:19:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.664+0000 7fb5014e8640 1 -- 172.21.15.31:0/3545045340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4fc1a06f0 con 0x7fb4fc105360 2024-04-25T13:19:25.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.665+0000 7fb5014e8640 1 -- 172.21.15.31:0/3545045340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4fc0fdf30 con 0x7fb4fc105360 2024-04-25T13:19:25.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.666+0000 7fb4f3fff640 1 -- 172.21.15.31:0/3545045340 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb4e4017070 con 0x7fb4fc105360 2024-04-25T13:19:25.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.667+0000 7fb4f3fff640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3d50 0x7fb4e80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.667+0000 7fb4faffd640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3d50 0x7fb4e80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.667+0000 7fb4f3fff640 1 -- 172.21.15.31:0/3545045340 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fb4e40cfef0 con 0x7fb4fc105360 2024-04-25T13:19:25.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.667+0000 7fb4faffd640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3d50 0x7fb4e80a6210 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4031500 tx=0x7fb4f4005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:25.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.676+0000 7fb4f3fff640 1 -- 172.21.15.31:0/3545045340 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4e401e030 con 0x7fb4fc105360 2024-04-25T13:19:25.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.865+0000 7fb5014e8640 1 -- 172.21.15.31:0/3545045340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb4fc062b50 con 0x7fb4fc105360 2024-04-25T13:19:25.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.865+0000 7fb4f3fff640 1 -- 172.21.15.31:0/3545045340 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fb4e409c8b0 con 0x7fb4fc105360 2024-04-25T13:19:25.865 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T13:19:25.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 -- 172.21.15.31:0/3545045340 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3d50 msgr2=0x7fb4e80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3d50 0x7fb4e80a6210 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4031500 tx=0x7fb4f4005a20 comp rx=0 tx=0).stop 2024-04-25T13:19:25.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 -- 172.21.15.31:0/3545045340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc105360 msgr2=0x7fb4fc1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc105360 0x7fb4fc1987f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400c970 tx=0x7fb4e400ce30 comp rx=0 tx=0).stop 2024-04-25T13:19:25.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 -- 172.21.15.31:0/3545045340 shutdown_connections 2024-04-25T13:19:25.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fb4e80a3d50 0x7fb4e80a6210 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc107c90 0x7fb4fc19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4fc105360 0x7fb4fc1987f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 --2- 172.21.15.31:0/3545045340 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb4fc102a30 0x7fb4fc1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 -- 172.21.15.31:0/3545045340 >> 172.21.15.31:0/3545045340 conn(0x7fb4fc0fc440 msgr2=0x7fb4fc103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:25.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 -- 172.21.15.31:0/3545045340 shutdown_connections 2024-04-25T13:19:25.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.867+0000 7fb5014e8640 1 -- 172.21.15.31:0/3545045340 wait complete. 2024-04-25T13:19:25.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:19:25.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.983+0000 7fd35ef44640 1 -- 172.21.15.31:0/3109806617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358101fe0 msgr2=0x7fd358102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.983+0000 7fd35ef44640 1 --2- 172.21.15.31:0/3109806617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358101fe0 0x7fd358102460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd344009900 tx=0x7fd34402a410 comp rx=0 tx=0).stop 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.983+0000 7fd35ef44640 1 -- 172.21.15.31:0/3109806617 shutdown_connections 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.983+0000 7fd35ef44640 1 --2- 172.21.15.31:0/3109806617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3581029a0 0x7fd3581092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.983+0000 7fd35ef44640 1 --2- 172.21.15.31:0/3109806617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358101fe0 0x7fd358102460 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.983+0000 7fd35ef44640 1 --2- 172.21.15.31:0/3109806617 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd358100e90 0x7fd3581012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.983+0000 7fd35ef44640 1 -- 172.21.15.31:0/3109806617 >> 172.21.15.31:0/3109806617 conn(0x7fd3580fc400 msgr2=0x7fd3580fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.983+0000 7fd35ef44640 1 -- 172.21.15.31:0/3109806617 shutdown_connections 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.983+0000 7fd35ef44640 1 -- 172.21.15.31:0/3109806617 wait complete. 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd35ef44640 1 Processor -- start 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd35ef44640 1 -- start start 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd35ef44640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd358100e90 0x7fd358069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd35ef44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358101fe0 0x7fd35819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd35ef44640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3581029a0 0x7fd35819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd35ef44640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810ca30 con 0x7fd358101fe0 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd35ef44640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810c8b0 con 0x7fd358100e90 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd35ef44640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810cbb0 con 0x7fd3581029a0 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd357fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358101fe0 0x7fd35819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd357fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358101fe0 0x7fd35819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34692/0 (socket says 172.21.15.31:34692) 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd357fff640 1 -- 172.21.15.31:0/577456990 learned_addr learned my addr 172.21.15.31:0/577456990 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd35cce0640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd358100e90 0x7fd358069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.984+0000 7fd35d4e1640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3581029a0 0x7fd35819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd357fff640 1 -- 172.21.15.31:0/577456990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3581029a0 msgr2=0x7fd35819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd357fff640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3581029a0 0x7fd35819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd357fff640 1 -- 172.21.15.31:0/577456990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd358100e90 msgr2=0x7fd358069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd357fff640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd358100e90 0x7fd358069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:25.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd357fff640 1 -- 172.21.15.31:0/577456990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35819fe00 con 0x7fd358101fe0 2024-04-25T13:19:25.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd35d4e1640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3581029a0 0x7fd35819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:25.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd35cce0640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd358100e90 0x7fd358069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:19:25.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd357fff640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358101fe0 0x7fd35819cf00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd344031500 tx=0x7fd344003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:25.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd355ffb640 1 -- 172.21.15.31:0/577456990 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34403f070 con 0x7fd358101fe0 2024-04-25T13:19:25.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd355ffb640 1 -- 172.21.15.31:0/577456990 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd344031c50 con 0x7fd358101fe0 2024-04-25T13:19:25.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd35ef44640 1 -- 172.21.15.31:0/577456990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3581a0090 con 0x7fd358101fe0 2024-04-25T13:19:25.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd355ffb640 1 -- 172.21.15.31:0/577456990 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344035030 con 0x7fd358101fe0 2024-04-25T13:19:25.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.985+0000 7fd35ef44640 1 -- 172.21.15.31:0/577456990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3581a0580 con 0x7fd358101fe0 2024-04-25T13:19:25.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.986+0000 7fd35ef44640 1 -- 172.21.15.31:0/577456990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd358108220 con 0x7fd358101fe0 2024-04-25T13:19:25.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.987+0000 7fd355ffb640 1 -- 172.21.15.31:0/577456990 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd344042070 con 0x7fd358101fe0 2024-04-25T13:19:25.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.988+0000 7fd355ffb640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd3480a3bb0 0x7fd3480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:25.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.988+0000 7fd355ffb640 1 -- 172.21.15.31:0/577456990 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fd3440f12a0 con 0x7fd358101fe0 2024-04-25T13:19:25.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.988+0000 7fd35cce0640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd3480a3bb0 0x7fd3480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:25.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.989+0000 7fd35cce0640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd3480a3bb0 0x7fd3480a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009fc0 tx=0x7fd34c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:25.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:25.992+0000 7fd355ffb640 1 -- 172.21.15.31:0/577456990 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3440bdc60 con 0x7fd358101fe0 2024-04-25T13:19:26.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:25 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2492656845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:26.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:25 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/720870967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:26.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:25 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3545045340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:26.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.221+0000 7fd35ef44640 1 -- 172.21.15.31:0/577456990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd35802d5c0 con 0x7fd358101fe0 2024-04-25T13:19:26.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.221+0000 7fd355ffb640 1 -- 172.21.15.31:0/577456990 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11798 (secure 0 0 0) 0x7fd3440bd560 con 0x7fd358101fe0 2024-04-25T13:19:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:26.222 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2599371925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2599371925},{"type":"v1","addr":"172.21.15.31:6839","nonce":2599371925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/164386980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":164386980},{"type":"v1","addr":"172.21.15.31:6841","nonce":164386980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:17:40.162672+0000","modified":"2024-04-25T13:17:44.194940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2682191586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2682191586},{"type":"v1","addr":"172.21.15.66:6839","nonce":2682191586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24511]},"id":1}]} 2024-04-25T13:19:26.222 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-04-25T13:19:26.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 -- 172.21.15.31:0/577456990 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd3480a3bb0 msgr2=0x7fd3480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd3480a3bb0 0x7fd3480a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009fc0 tx=0x7fd34c005f90 comp rx=0 tx=0).stop 2024-04-25T13:19:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 -- 172.21.15.31:0/577456990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358101fe0 msgr2=0x7fd35819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358101fe0 0x7fd35819cf00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd344031500 tx=0x7fd344003c10 comp rx=0 tx=0).stop 2024-04-25T13:19:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 -- 172.21.15.31:0/577456990 shutdown_connections 2024-04-25T13:19:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fd3480a3bb0 0x7fd3480a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3581029a0 0x7fd35819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd358101fe0 0x7fd35819cf00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 --2- 172.21.15.31:0/577456990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd358100e90 0x7fd358069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 -- 172.21.15.31:0/577456990 >> 172.21.15.31:0/577456990 conn(0x7fd3580fc400 msgr2=0x7fd3581053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 -- 172.21.15.31:0/577456990 shutdown_connections 2024-04-25T13:19:26.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.224+0000 7fd35ef44640 1 -- 172.21.15.31:0/577456990 wait complete. 2024-04-25T13:19:26.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:25 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2492656845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:26.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:25 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/720870967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:26.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:25 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3545045340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:26.235 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:19:26.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.338+0000 7f6fa90e1640 1 -- 172.21.15.31:0/634970685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa41029a0 msgr2=0x7f6fa41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.338+0000 7f6fa90e1640 1 --2- 172.21.15.31:0/634970685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa41029a0 0x7f6fa41092c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009ca0 tx=0x7f6f9c02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.338+0000 7f6fa90e1640 1 -- 172.21.15.31:0/634970685 shutdown_connections 2024-04-25T13:19:26.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.338+0000 7f6fa90e1640 1 --2- 172.21.15.31:0/634970685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa41029a0 0x7f6fa41092c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.338+0000 7f6fa90e1640 1 --2- 172.21.15.31:0/634970685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fa4101fe0 0x7f6fa4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.338+0000 7f6fa90e1640 1 --2- 172.21.15.31:0/634970685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fa4100e90 0x7f6fa41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.338+0000 7f6fa90e1640 1 -- 172.21.15.31:0/634970685 >> 172.21.15.31:0/634970685 conn(0x7f6fa40fc400 msgr2=0x7f6fa40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.338+0000 7f6fa90e1640 1 -- 172.21.15.31:0/634970685 shutdown_connections 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.338+0000 7f6fa90e1640 1 -- 172.21.15.31:0/634970685 wait complete. 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.338+0000 7f6fa90e1640 1 Processor -- start 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa90e1640 1 -- start start 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa90e1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fa4100e90 0x7f6fa4104da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa90e1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fa4101fe0 0x7f6fa4103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa90e1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa41029a0 0x7f6fa4103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa90e1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410ca40 con 0x7f6fa41029a0 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa90e1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410c8c0 con 0x7f6fa4101fe0 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa90e1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410cbc0 con 0x7f6fa4100e90 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa41029a0 0x7f6fa4103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa41029a0 0x7f6fa4103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34714/0 (socket says 172.21.15.31:34714) 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa359e640 1 -- 172.21.15.31:0/956384939 learned_addr learned my addr 172.21.15.31:0/956384939 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:26.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa259c640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fa4101fe0 0x7f6fa4103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa2d9d640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fa4100e90 0x7f6fa4104da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa359e640 1 -- 172.21.15.31:0/956384939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fa4100e90 msgr2=0x7f6fa4104da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa359e640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fa4100e90 0x7f6fa4104da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa359e640 1 -- 172.21.15.31:0/956384939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fa4101fe0 msgr2=0x7f6fa4103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa359e640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fa4101fe0 0x7f6fa4103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa359e640 1 -- 172.21.15.31:0/956384939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa41041b0 con 0x7f6fa41029a0 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa2d9d640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fa4100e90 0x7f6fa4104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa259c640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fa4101fe0 0x7f6fa4103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.339+0000 7f6fa359e640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa41029a0 0x7f6fa4103990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c033040 tx=0x7f6f9c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.340+0000 7f6f8bfff640 1 -- 172.21.15.31:0/956384939 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0317c0 con 0x7f6fa41029a0 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.340+0000 7f6f8bfff640 1 -- 172.21.15.31:0/956384939 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c0044f0 con 0x7f6fa41029a0 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.340+0000 7f6fa90e1640 1 -- 172.21.15.31:0/956384939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa41a2240 con 0x7f6fa41029a0 2024-04-25T13:19:26.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.340+0000 7f6f8bfff640 1 -- 172.21.15.31:0/956384939 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c02fd00 con 0x7f6fa41029a0 2024-04-25T13:19:26.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.340+0000 7f6fa90e1640 1 -- 172.21.15.31:0/956384939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa41a2790 con 0x7f6fa41029a0 2024-04-25T13:19:26.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.341+0000 7f6fa90e1640 1 -- 172.21.15.31:0/956384939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa4068d70 con 0x7f6fa41029a0 2024-04-25T13:19:26.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.342+0000 7f6f8bfff640 1 -- 172.21.15.31:0/956384939 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6f9c004040 con 0x7f6fa41029a0 2024-04-25T13:19:26.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.343+0000 7f6f8bfff640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6f8c0a3bb0 0x7f6f8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.343+0000 7f6fa2d9d640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6f8c0a3bb0 0x7f6f8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.343+0000 7f6f8bfff640 1 -- 172.21.15.31:0/956384939 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f6f9c0f1c80 con 0x7f6fa41029a0 2024-04-25T13:19:26.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.344+0000 7f6fa2d9d640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6f8c0a3bb0 0x7f6f8c0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6f980058c0 tx=0x7f6f98005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:26.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.347+0000 7f6f8bfff640 1 -- 172.21.15.31:0/956384939 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f9c0be640 con 0x7f6fa41029a0 2024-04-25T13:19:26.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:25 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2492656845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:19:26.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:25 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/720870967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:19:26.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:25 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3545045340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:19:26.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.535+0000 7f6fa90e1640 1 -- 172.21.15.31:0/956384939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fa41a2bb0 con 0x7f6fa41029a0 2024-04-25T13:19:26.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.537+0000 7f6f8bfff640 1 -- 172.21.15.31:0/956384939 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22650 (secure 0 0 0) 0x7f6f9c0bdf40 con 0x7f6fa41029a0 2024-04-25T13:19:26.537 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:19:26.537 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":90,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:18:32.938284+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.2e","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:19:26.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 -- 172.21.15.31:0/956384939 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6f8c0a3bb0 msgr2=0x7f6f8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6f8c0a3bb0 0x7f6f8c0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6f980058c0 tx=0x7f6f98005d30 comp rx=0 tx=0).stop 2024-04-25T13:19:26.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 -- 172.21.15.31:0/956384939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa41029a0 msgr2=0x7f6fa4103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa41029a0 0x7f6fa4103990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c033040 tx=0x7f6f9c003ba0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 -- 172.21.15.31:0/956384939 shutdown_connections 2024-04-25T13:19:26.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f6f8c0a3bb0 0x7f6f8c0a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fa41029a0 0x7f6fa4103990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6fa4101fe0 0x7f6fa4103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 --2- 172.21.15.31:0/956384939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6fa4100e90 0x7f6fa4104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 -- 172.21.15.31:0/956384939 >> 172.21.15.31:0/956384939 conn(0x7f6fa40fc400 msgr2=0x7f6fa40fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:26.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 -- 172.21.15.31:0/956384939 shutdown_connections 2024-04-25T13:19:26.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.539+0000 7f6fa90e1640 1 -- 172.21.15.31:0/956384939 wait complete. 2024-04-25T13:19:26.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:17:40.162672+0000', 'modified': '2024-04-25T13:17:44.194940+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24511}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.66:6839/2682191586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6838', 'nonce': 2682191586}, {'type': 'v1', 'addr': '172.21.15.66:6839', 'nonce': 2682191586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.136:6839/2196424437', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 2196424437}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 2196424437}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24511, 'qdb_cluster': [24511]} 2024-04-25T13:19:26.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:19:26.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T13:19:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.654+0000 7f33e7689640 1 -- 172.21.15.31:0/1545603625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33e0107d10 msgr2=0x7f33e010a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.654+0000 7f33e7689640 1 --2- 172.21.15.31:0/1545603625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33e0107d10 0x7f33e010a100 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009ca0 tx=0x7f33d802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.654+0000 7f33e7689640 1 -- 172.21.15.31:0/1545603625 shutdown_connections 2024-04-25T13:19:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.654+0000 7f33e7689640 1 --2- 172.21.15.31:0/1545603625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33e0107d10 0x7f33e010a100 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.654+0000 7f33e7689640 1 --2- 172.21.15.31:0/1545603625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33e01053e0 0x7f33e01077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.654+0000 7f33e7689640 1 --2- 172.21.15.31:0/1545603625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e0102ab0 0x7f33e0104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.654+0000 7f33e7689640 1 -- 172.21.15.31:0/1545603625 >> 172.21.15.31:0/1545603625 conn(0x7f33e00fc440 msgr2=0x7f33e00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.654+0000 7f33e7689640 1 -- 172.21.15.31:0/1545603625 shutdown_connections 2024-04-25T13:19:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.654+0000 7f33e7689640 1 -- 172.21.15.31:0/1545603625 wait complete. 2024-04-25T13:19:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e7689640 1 Processor -- start 2024-04-25T13:19:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e7689640 1 -- start start 2024-04-25T13:19:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e7689640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33e0102ab0 0x7f33e0198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e7689640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e01053e0 0x7f33e01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e7689640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33e0107d10 0x7f33e019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e7689640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f33e010daf0 con 0x7f33e0107d10 2024-04-25T13:19:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e7689640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33e010d970 con 0x7f33e01053e0 2024-04-25T13:19:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e7689640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33e010dc70 con 0x7f33e0102ab0 2024-04-25T13:19:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e5c26640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33e0107d10 0x7f33e019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e5c26640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33e0107d10 0x7f33e019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34736/0 (socket says 172.21.15.31:34736) 2024-04-25T13:19:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e5c26640 1 -- 172.21.15.31:0/4154182543 learned_addr learned my addr 172.21.15.31:0/4154182543 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e5425640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33e0102ab0 0x7f33e0198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e4c24640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e01053e0 0x7f33e01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e5c26640 1 -- 172.21.15.31:0/4154182543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33e0102ab0 msgr2=0x7f33e0198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e5c26640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33e0102ab0 0x7f33e0198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e5c26640 1 -- 172.21.15.31:0/4154182543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e01053e0 msgr2=0x7f33e01986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e5c26640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e01053e0 0x7f33e01986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.655+0000 7f33e5c26640 1 -- 172.21.15.31:0/4154182543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e019fd80 con 0x7f33e0107d10 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.656+0000 7f33e5425640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33e0102ab0 0x7f33e0198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.656+0000 7f33e5c26640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33e0107d10 0x7f33e019f710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f33d80097b0 tx=0x7f33d8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.656+0000 7f33d67fc640 1 -- 172.21.15.31:0/4154182543 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8004240 con 0x7f33e0107d10 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.656+0000 7f33d67fc640 1 -- 172.21.15.31:0/4154182543 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d80317c0 con 0x7f33e0107d10 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.656+0000 7f33e7689640 1 -- 172.21.15.31:0/4154182543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e01a0010 con 0x7f33e0107d10 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.656+0000 7f33d67fc640 1 -- 172.21.15.31:0/4154182543 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d802fd80 con 0x7f33e0107d10 2024-04-25T13:19:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.656+0000 7f33e7689640 1 -- 172.21.15.31:0/4154182543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e01a05b0 con 0x7f33e0107d10 2024-04-25T13:19:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.658+0000 7f33d67fc640 1 -- 172.21.15.31:0/4154182543 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f33d80043e0 con 0x7f33e0107d10 2024-04-25T13:19:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33ea39b740 1 Processor -- start 2024-04-25T13:19:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33ea39b740 1 -- start start 2024-04-25T13:19:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33d67fc640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f33cc0a3c30 0x7f33cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08373b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b083f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33ea39b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5600b0797c50 con 0x5600b0836f90 2024-04-25T13:19:26.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33ea39b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5600b0734830 con 0x5600b0838160 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33ea39b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5600b0793c10 con 0x5600b0838b20 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e5425640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e4c24640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f33cc0a3c30 0x7f33cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e5425640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08373b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34752/0 (socket says 172.21.15.31:34752) 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e5425640 1 -- 172.21.15.31:0/2874881670 learned_addr learned my addr 172.21.15.31:0/2874881670 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33d67fc640 1 -- 172.21.15.31:0/4154182543 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f33d80f1d00 con 0x7f33e0107d10 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e5c26640 1 --2- 172.21.15.31:0/2874881670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b083f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e4c24640 1 --2- 172.21.15.31:0/2874881670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e5425640 1 -- 172.21.15.31:0/2874881670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 msgr2=0x5600b083f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e5425640 1 --2- 172.21.15.31:0/2874881670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b083f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e5425640 1 -- 172.21.15.31:0/2874881670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 msgr2=0x5600b08385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e5425640 1 --2- 172.21.15.31:0/2874881670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e5425640 1 -- 172.21.15.31:0/2874881670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b04aa2d0 con 0x5600b0836f90 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e5c26640 1 --2- 172.21.15.31:0/2874881670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b083f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e4c24640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f33cc0a3c30 0x7f33cc0a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f33e0199670 tx=0x7f33d0014840 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.659+0000 7f33e5425640 1 --2- 172.21.15.31:0/2874881670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08373b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009c50 tx=0x7f33dc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ee4a72f4fccedc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33cb7fe640 1 -- 172.21.15.31:0/2874881670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc016070 con 0x5600b0836f90 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33cb7fe640 1 -- 172.21.15.31:0/2874881670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33dc0054d0 con 0x5600b0836f90 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33cb7fe640 1 -- 172.21.15.31:0/2874881670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc011470 con 0x5600b0836f90 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33ea39b740 1 -- 172.21.15.31:0/2874881670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 msgr2=0x5600b08373b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33ea39b740 1 --2- 172.21.15.31:0/2874881670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08373b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009c50 tx=0x7f33dc00e6c0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33ea39b740 1 -- 172.21.15.31:0/2874881670 shutdown_connections 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33ea39b740 1 --2- 172.21.15.31:0/2874881670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b083f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33ea39b740 1 --2- 172.21.15.31:0/2874881670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33ea39b740 1 --2- 172.21.15.31:0/2874881670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08373b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33ea39b740 1 -- 172.21.15.31:0/2874881670 >> 172.21.15.31:0/2874881670 conn(0x5600b082e070 msgr2=0x5600b082e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33ea39b740 1 -- 172.21.15.31:0/2874881670 shutdown_connections 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.660+0000 7f33ea39b740 1 -- 172.21.15.31:0/2874881670 wait complete. 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33ea39b740 1 -- start start 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33ea39b740 1 Processor -- start 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b0848930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33ea39b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5600b083fb30 con 0x5600b0836f90 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33ea39b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5600b0732470 con 0x5600b0838160 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33ea39b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5600b083fcb0 con 0x5600b0838b20 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33ea39b740 10 client.0 _ll_register_callbacks cb 0x5600b0757f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33e5425640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33e5425640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34764/0 (socket says 172.21.15.31:34764) 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33e5425640 1 -- 172.21.15.31:0/4048245381 learned_addr learned my addr 172.21.15.31:0/4048245381 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33e5c26640 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b0848930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33e4c24640 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33e5425640 1 -- 172.21.15.31:0/4048245381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 msgr2=0x5600b0848930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33e5425640 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b0848930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33e5425640 1 -- 172.21.15.31:0/4048245381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 msgr2=0x5600b08418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.661+0000 7f33e5425640 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33e5425640 1 -- 172.21.15.31:0/4048245381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b05746b0 con 0x5600b0836f90 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33e5425640 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08413b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f33dc005430 tx=0x7f33dc005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abff97358268fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33c8ff9640 1 -- 172.21.15.31:0/4048245381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc028060 con 0x5600b0836f90 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33c8ff9640 1 -- 172.21.15.31:0/4048245381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f33dc009d40 con 0x5600b0836f90 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33ea39b740 1 -- 172.21.15.31:0/4048245381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33dc0054d0 con 0x5600b0836f90 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33c8ff9640 1 -- 172.21.15.31:0/4048245381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f33dc016040 con 0x5600b0836f90 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33ea39b740 10 client.15174 fetch_fsmap learned FSMap version 20 2024-04-25T13:19:26.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33ea39b740 1 -- 172.21.15.31:0/4048245381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5600b084bc80 con 0x5600b0836f90 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33c8ff9640 1 -- 172.21.15.31:0/4048245381 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33dc0054d0 con 0x5600b0836f90 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33c8ff9640 1 -- 172.21.15.31:0/4048245381 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f33dc00f040 con 0x5600b0836f90 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33ea39b740 10 client.15174 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33ea39b740 10 client.15174 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33ea39b740 20 client.15174 populate_metadata read hostname 'smithi031' 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33ea39b740 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5600b084df70 0x5600b084f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33ea39b740 4 client.15174 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.662+0000 7f33ea39b740 1 -- 172.21.15.31:0/4048245381 --> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600b07a5670 con 0x5600b084df70 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.663+0000 7f33e4c24640 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5600b084df70 0x5600b084f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.663+0000 7f33e4c24640 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5600b084df70 0x5600b084f3e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1529497c3ec2324 server_cookie=d7e527cf2ba01e6d in_seq=0 out_seq=0 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.663+0000 7f33c8ff9640 10 client.15174 ms_handle_connect on v2:172.21.15.136:6838/2196424437 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33c8ff9640 1 -- 172.21.15.31:0/4048245381 <== mds.0 v2:172.21.15.136:6838/2196424437 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600b07a5670 con 0x5600b084df70 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33c8ff9640 10 client.15174 handle_command_reply: tid=0 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 1 client.15174 shutdown 2024-04-25T13:19:26.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 1 -- 172.21.15.31:0/4048245381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 msgr2=0x5600b08413b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08413b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f33dc005430 tx=0x7f33dc005460 comp rx=0 tx=0).stop 2024-04-25T13:19:26.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 1 -- 172.21.15.31:0/4048245381 shutdown_connections 2024-04-25T13:19:26.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5600b084df70 0x5600b084f3e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b0848930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33c8ff9640 0 client.15174 ms_handle_reset on v2:172.21.15.136:6838/2196424437 2024-04-25T13:19:26.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 1 --2- 172.21.15.31:0/4048245381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b08413b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 1 -- 172.21.15.31:0/4048245381 >> 172.21.15.31:0/4048245381 conn(0x5600b082e070 msgr2=0x5600b082d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:26.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 1 -- 172.21.15.31:0/4048245381 shutdown_connections 2024-04-25T13:19:26.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 1 -- 172.21.15.31:0/4048245381 wait complete. 2024-04-25T13:19:26.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.664+0000 7f33ea39b740 20 client.15174 trim_cache size 0 max 16384 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.682+0000 7f33ea39b740 1 Processor -- start 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.682+0000 7f33ea39b740 1 -- start start 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b0830f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b0848850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b0831490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33ea39b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5600b084d720 con 0x5600b0836f90 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33ea39b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5600b084ddf0 con 0x5600b0838160 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33ea39b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5600b084e4c0 con 0x5600b0838b20 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e5425640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b0830f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e5425640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b0830f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34768/0 (socket says 172.21.15.31:34768) 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e5425640 1 -- 172.21.15.31:0/313152061 learned_addr learned my addr 172.21.15.31:0/313152061 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e5c26640 1 --2- 172.21.15.31:0/313152061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b0831490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e4c24640 1 --2- 172.21.15.31:0/313152061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b0848850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e5425640 1 -- 172.21.15.31:0/313152061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 msgr2=0x5600b0831490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e5425640 1 --2- 172.21.15.31:0/313152061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b0831490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e5425640 1 -- 172.21.15.31:0/313152061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 msgr2=0x5600b0848850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e5425640 1 --2- 172.21.15.31:0/313152061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b0848850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e5425640 1 -- 172.21.15.31:0/313152061 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b082dea0 con 0x5600b0836f90 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e5c26640 1 --2- 172.21.15.31:0/313152061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b0831490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e4c24640 1 --2- 172.21.15.31:0/313152061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b0848850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33e5425640 1 --2- 172.21.15.31:0/313152061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b0830f50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f33dc005000 tx=0x7f33dc009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9029541ad4dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33caffd640 1 -- 172.21.15.31:0/313152061 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc032030 con 0x5600b0836f90 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33caffd640 1 -- 172.21.15.31:0/313152061 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33dc009d40 con 0x5600b0836f90 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.683+0000 7f33caffd640 1 -- 172.21.15.31:0/313152061 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc028040 con 0x5600b0836f90 2024-04-25T13:19:26.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.684+0000 7f33ea39b740 1 -- 172.21.15.31:0/313152061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 msgr2=0x5600b0830f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.684+0000 7f33ea39b740 1 --2- 172.21.15.31:0/313152061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b0830f50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f33dc005000 tx=0x7f33dc009560 comp rx=0 tx=0).stop 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.684+0000 7f33ea39b740 1 -- 172.21.15.31:0/313152061 shutdown_connections 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.684+0000 7f33ea39b740 1 --2- 172.21.15.31:0/313152061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0838b20 0x5600b0831490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.684+0000 7f33ea39b740 1 --2- 172.21.15.31:0/313152061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b0848850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.684+0000 7f33ea39b740 1 --2- 172.21.15.31:0/313152061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0836f90 0x5600b0830f50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.684+0000 7f33ea39b740 1 -- 172.21.15.31:0/313152061 >> 172.21.15.31:0/313152061 conn(0x5600b082e070 msgr2=0x5600b083bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.684+0000 7f33ea39b740 1 -- 172.21.15.31:0/313152061 shutdown_connections 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.684+0000 7f33ea39b740 1 -- 172.21.15.31:0/313152061 wait complete. 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.684+0000 7f33ea39b740 1 -- start start 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.684+0000 7f33ea39b740 1 Processor -- start 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0836f90 0x5600b0844290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08447d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33ea39b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0838b20 0x5600b0847780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33ea39b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5600b084f260 con 0x5600b0838b20 2024-04-25T13:19:26.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33ea39b740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5600b084eb90 con 0x5600b0838160 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33ea39b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5600b084f930 con 0x5600b0836f90 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33ea39b740 10 client.0 _ll_register_callbacks cb 0x7f33dc009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e5c26640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0838b20 0x5600b0847780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e5c26640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0838b20 0x5600b0847780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34774/0 (socket says 172.21.15.31:34774) 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e5c26640 1 -- 172.21.15.31:0/316744826 learned_addr learned my addr 172.21.15.31:0/316744826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e5425640 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0836f90 0x5600b0844290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e4c24640 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e5c26640 1 -- 172.21.15.31:0/316744826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0836f90 msgr2=0x5600b0844290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e5c26640 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0836f90 0x5600b0844290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e5c26640 1 -- 172.21.15.31:0/316744826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 msgr2=0x5600b08447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e5c26640 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e5c26640 1 -- 172.21.15.31:0/316744826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b0847d70 con 0x5600b0838b20 2024-04-25T13:19:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e4c24640 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33e5c26640 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0838b20 0x5600b0847780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f33d80bdcc0 tx=0x7f33d8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192cc66d307bbfa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33c8ff9640 1 -- 172.21.15.31:0/316744826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d80be6c0 con 0x5600b0838b20 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33c8ff9640 1 -- 172.21.15.31:0/316744826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f33d804dcd0 con 0x5600b0838b20 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33ea39b740 1 -- 172.21.15.31:0/316744826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33dc009d40 con 0x5600b0838b20 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33c8ff9640 1 -- 172.21.15.31:0/316744826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f33d80491c0 con 0x5600b0838b20 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33ea39b740 10 client.15180 fetch_fsmap learned FSMap version 20 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.685+0000 7f33ea39b740 1 -- 172.21.15.31:0/316744826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5600b08b4130 con 0x5600b0838b20 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.686+0000 7f33c8ff9640 1 -- 172.21.15.31:0/316744826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33dc009d40 con 0x5600b0838b20 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.686+0000 7f33c8ff9640 1 -- 172.21.15.31:0/316744826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f33d80c0070 con 0x5600b0838b20 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.686+0000 7f33ea39b740 10 client.15180 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.686+0000 7f33ea39b740 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.686+0000 7f33ea39b740 20 client.15180 populate_metadata read hostname 'smithi031' 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.686+0000 7f33ea39b740 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5600b084df70 0x5600b08b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.686+0000 7f33ea39b740 4 client.15180 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.686+0000 7f33ea39b740 1 -- 172.21.15.31:0/316744826 --> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] -- command(tid 0: {"prefix": "status"}) -- 0x5600b079f4e0 con 0x5600b084df70 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.686+0000 7f33e5425640 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5600b084df70 0x5600b08b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.686+0000 7f33c8ff9640 10 client.15180 ms_handle_connect on v2:172.21.15.136:6838/2196424437 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.686+0000 7f33e5425640 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5600b084df70 0x5600b08b7b80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ba0da8fac58f6bf server_cookie=840ab425a5499653 in_seq=0 out_seq=0 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33c8ff9640 1 -- 172.21.15.31:0/316744826 <== mds.0 v2:172.21.15.136:6838/2196424437 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5600b079f4e0 con 0x5600b084df70 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33c8ff9640 10 client.15180 handle_command_reply: tid=0 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 1 client.15180 shutdown 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 1 -- 172.21.15.31:0/316744826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0838b20 msgr2=0x5600b0847780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0838b20 0x5600b0847780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f33d80bdcc0 tx=0x7f33d8004790 comp rx=0 tx=0).stop 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 1 -- 172.21.15.31:0/316744826 shutdown_connections 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.136:6838/2196424437,v1:172.21.15.136:6839/2196424437] conn(0x5600b084df70 0x5600b08b7b80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600b0838b20 0x5600b0847780 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33c8ff9640 0 client.15180 ms_handle_reset on v2:172.21.15.136:6838/2196424437 2024-04-25T13:19:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5600b0838160 0x5600b08447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 1 --2- 172.21.15.31:0/316744826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600b0836f90 0x5600b0844290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 1 -- 172.21.15.31:0/316744826 >> 172.21.15.31:0/316744826 conn(0x5600b082e070 msgr2=0x5600b082c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 1 -- 172.21.15.31:0/316744826 shutdown_connections 2024-04-25T13:19:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 1 -- 172.21.15.31:0/316744826 wait complete. 2024-04-25T13:19:26.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.687+0000 7f33ea39b740 20 client.15180 trim_cache size 0 max 16384 2024-04-25T13:19:26.695 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T13:19:26.695 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "e4b023f4-0304-11ef-bc93-c7b262605968", 2024-04-25T13:19:26.696 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-04-25T13:19:26.696 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24511, 2024-04-25T13:19:26.696 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-04-25T13:19:26.696 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-04-25T13:19:26.696 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-04-25T13:19:26.696 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 105.886284895, 2024-04-25T13:19:26.696 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 20, 2024-04-25T13:19:26.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 90, 2024-04-25T13:19:26.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T13:19:26.696 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 119.489707389 2024-04-25T13:19:26.696 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T13:19:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 -- 172.21.15.31:0/4154182543 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f33cc0a3c30 msgr2=0x7f33cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f33cc0a3c30 0x7f33cc0a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f33e0199670 tx=0x7f33d0014840 comp rx=0 tx=0).stop 2024-04-25T13:19:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 -- 172.21.15.31:0/4154182543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33e0107d10 msgr2=0x7f33e019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33e0107d10 0x7f33e019f710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f33d80097b0 tx=0x7f33d8003c20 comp rx=0 tx=0).stop 2024-04-25T13:19:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 -- 172.21.15.31:0/4154182543 shutdown_connections 2024-04-25T13:19:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f33cc0a3c30 0x7f33cc0a60f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33e0107d10 0x7f33e019f710 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33e01053e0 0x7f33e01986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 --2- 172.21.15.31:0/4154182543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33e0102ab0 0x7f33e0198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 -- 172.21.15.31:0/4154182543 >> 172.21.15.31:0/4154182543 conn(0x7f33e00fc440 msgr2=0x7f33e0103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 -- 172.21.15.31:0/4154182543 shutdown_connections 2024-04-25T13:19:26.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.688+0000 7f33e7689640 1 -- 172.21.15.31:0/4154182543 wait complete. 2024-04-25T13:19:26.699 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T13:19:26.708 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T13:19:26.709 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T13:19:26.709 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T13:19:26.709 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T13:19:26.710 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:19:26.710 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T13:19:26.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T13:19:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.812+0000 7fcd95f15640 1 -- 172.21.15.31:0/248393645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd90107d30 msgr2=0x7fcd9010a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.812+0000 7fcd95f15640 1 --2- 172.21.15.31:0/248393645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd90107d30 0x7fcd9010a180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009ca0 tx=0x7fcd8802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.812+0000 7fcd95f15640 1 -- 172.21.15.31:0/248393645 shutdown_connections 2024-04-25T13:19:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.812+0000 7fcd95f15640 1 --2- 172.21.15.31:0/248393645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd90107d30 0x7fcd9010a180 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.812+0000 7fcd95f15640 1 --2- 172.21.15.31:0/248393645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd90105400 0x7fcd901077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.812+0000 7fcd95f15640 1 --2- 172.21.15.31:0/248393645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd90102ad0 0x7fcd90104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.812+0000 7fcd95f15640 1 -- 172.21.15.31:0/248393645 >> 172.21.15.31:0/248393645 conn(0x7fcd900fc4c0 msgr2=0x7fcd900fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.812+0000 7fcd95f15640 1 -- 172.21.15.31:0/248393645 shutdown_connections 2024-04-25T13:19:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.812+0000 7fcd95f15640 1 -- 172.21.15.31:0/248393645 wait complete. 2024-04-25T13:19:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd95f15640 1 Processor -- start 2024-04-25T13:19:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd95f15640 1 -- start start 2024-04-25T13:19:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd95f15640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd90102ad0 0x7fcd901980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd95f15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd90105400 0x7fcd90198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd95f15640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd90107d30 0x7fcd9019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd95f15640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9010db20 con 0x7fcd90105400 2024-04-25T13:19:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd95f15640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9010d9a0 con 0x7fcd90107d30 2024-04-25T13:19:26.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd95f15640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9010dca0 con 0x7fcd90102ad0 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd8effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd90105400 0x7fcd90198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd8effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd90105400 0x7fcd90198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34786/0 (socket says 172.21.15.31:34786) 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd8effd640 1 -- 172.21.15.31:0/1251873866 learned_addr learned my addr 172.21.15.31:0/1251873866 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd8ffff640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd90107d30 0x7fcd9019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd8f7fe640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd90102ad0 0x7fcd901980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd8effd640 1 -- 172.21.15.31:0/1251873866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd90102ad0 msgr2=0x7fcd901980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd8effd640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd90102ad0 0x7fcd901980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd8effd640 1 -- 172.21.15.31:0/1251873866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd90107d30 msgr2=0x7fcd9019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd8effd640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd90107d30 0x7fcd9019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.813+0000 7fcd8effd640 1 -- 172.21.15.31:0/1251873866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9019fd00 con 0x7fcd90105400 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.814+0000 7fcd8effd640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd90105400 0x7fcd90198620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcd80005fd0 tx=0x7fcd8000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.814+0000 7fcd8cff9640 1 -- 172.21.15.31:0/1251873866 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80017070 con 0x7fcd90105400 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.814+0000 7fcd95f15640 1 -- 172.21.15.31:0/1251873866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd90100920 con 0x7fcd90105400 2024-04-25T13:19:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.814+0000 7fcd8cff9640 1 -- 172.21.15.31:0/1251873866 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd80009c30 con 0x7fcd90105400 2024-04-25T13:19:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.814+0000 7fcd8cff9640 1 -- 172.21.15.31:0/1251873866 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80003d30 con 0x7fcd90105400 2024-04-25T13:19:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.814+0000 7fcd95f15640 1 -- 172.21.15.31:0/1251873866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd90100e70 con 0x7fcd90105400 2024-04-25T13:19:26.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.815+0000 7fcd95f15640 1 -- 172.21.15.31:0/1251873866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd900fdfb0 con 0x7fcd90105400 2024-04-25T13:19:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.816+0000 7fcd8cff9640 1 -- 172.21.15.31:0/1251873866 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcd80003ed0 con 0x7fcd90105400 2024-04-25T13:19:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.816+0000 7fcd8cff9640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcd780a3c50 0x7fcd780a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.817+0000 7fcd8f7fe640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcd780a3c50 0x7fcd780a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.817+0000 7fcd8cff9640 1 -- 172.21.15.31:0/1251873866 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fcd800d0070 con 0x7fcd90105400 2024-04-25T13:19:26.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.817+0000 7fcd8f7fe640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcd780a3c50 0x7fcd780a6110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcd84005fd0 tx=0x7fcd8400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:26.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:26.821+0000 7fcd8cff9640 1 -- 172.21.15.31:0/1251873866 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd80020030 con 0x7fcd90105400 2024-04-25T13:19:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.060+0000 7fcd95f15640 1 -- 172.21.15.31:0/1251873866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fcd90102250 con 0x7fcd90105400 2024-04-25T13:19:27.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:26 smithi136 ceph-mon[33038]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:27.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:26 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/577456990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:27.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:26 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/956384939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:27.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:26 smithi031 ceph-mon[29457]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:27.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:26 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/577456990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:27.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:26 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/956384939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:26 smithi066 ceph-mon[33137]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:26 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/577456990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:19:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:26 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/956384939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:19:27.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.960+0000 7fcd8cff9640 1 -- 172.21.15.31:0/1251873866 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fcd80009de0 con 0x7fcd90105400 2024-04-25T13:19:27.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 -- 172.21.15.31:0/1251873866 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcd780a3c50 msgr2=0x7fcd780a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:27.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcd780a3c50 0x7fcd780a6110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcd84005fd0 tx=0x7fcd8400b390 comp rx=0 tx=0).stop 2024-04-25T13:19:27.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 -- 172.21.15.31:0/1251873866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd90105400 msgr2=0x7fcd90198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:27.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd90105400 0x7fcd90198620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcd80005fd0 tx=0x7fcd8000dde0 comp rx=0 tx=0).stop 2024-04-25T13:19:27.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 -- 172.21.15.31:0/1251873866 shutdown_connections 2024-04-25T13:19:27.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fcd780a3c50 0x7fcd780a6110 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:27.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcd90107d30 0x7fcd9019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:27.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd90105400 0x7fcd90198620 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:27.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 --2- 172.21.15.31:0/1251873866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd90102ad0 0x7fcd901980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:27.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 -- 172.21.15.31:0/1251873866 >> 172.21.15.31:0/1251873866 conn(0x7fcd900fc4c0 msgr2=0x7fcd90103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:27.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 -- 172.21.15.31:0/1251873866 shutdown_connections 2024-04-25T13:19:27.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:27.963+0000 7fcd95f15640 1 -- 172.21.15.31:0/1251873866 wait complete. 2024-04-25T13:19:27.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T13:19:28.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.093+0000 7fe8bad76640 1 -- 172.21.15.31:0/2265470141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b4100e60 msgr2=0x7fe8b4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:28.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.093+0000 7fe8bad76640 1 --2- 172.21.15.31:0/2265470141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b4100e60 0x7fe8b4101280 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009900 tx=0x7fe89c02a410 comp rx=0 tx=0).stop 2024-04-25T13:19:28.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.093+0000 7fe8bad76640 1 -- 172.21.15.31:0/2265470141 shutdown_connections 2024-04-25T13:19:28.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.093+0000 7fe8bad76640 1 --2- 172.21.15.31:0/2265470141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b4102a00 0x7fe8b4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.093+0000 7fe8bad76640 1 --2- 172.21.15.31:0/2265470141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8b4102040 0x7fe8b41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.093+0000 7fe8bad76640 1 --2- 172.21.15.31:0/2265470141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b4100e60 0x7fe8b4101280 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.093+0000 7fe8bad76640 1 -- 172.21.15.31:0/2265470141 >> 172.21.15.31:0/2265470141 conn(0x7fe8b40fc420 msgr2=0x7fe8b40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.093+0000 7fe8bad76640 1 -- 172.21.15.31:0/2265470141 shutdown_connections 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.093+0000 7fe8bad76640 1 -- 172.21.15.31:0/2265470141 wait complete. 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8bad76640 1 Processor -- start 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8bad76640 1 -- start start 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8bad76640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8b4100e60 0x7fe8b4197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8bad76640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b4102040 0x7fe8b41975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8bad76640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b4102a00 0x7fe8b419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8bad76640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b410ca40 con 0x7fe8b4102a00 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8bad76640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b410c8c0 con 0x7fe8b4100e60 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8bad76640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b410cbc0 con 0x7fe8b4102040 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8b9313640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b4102a00 0x7fe8b419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:28.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8b9313640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b4102a00 0x7fe8b419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34818/0 (socket says 172.21.15.31:34818) 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8b9313640 1 -- 172.21.15.31:0/1174064517 learned_addr learned my addr 172.21.15.31:0/1174064517 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.094+0000 7fe8b8b12640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8b4100e60 0x7fe8b4197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8b3fff640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b4102040 0x7fe8b41975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8b9313640 1 -- 172.21.15.31:0/1174064517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b4102040 msgr2=0x7fe8b41975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8b9313640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b4102040 0x7fe8b41975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8b9313640 1 -- 172.21.15.31:0/1174064517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8b4100e60 msgr2=0x7fe8b4197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8b9313640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8b4100e60 0x7fe8b4197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8b9313640 1 -- 172.21.15.31:0/1174064517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b419ecb0 con 0x7fe8b4102a00 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8b9313640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b4102a00 0x7fe8b419e640 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e970 tx=0x7fe8ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8b1ffb640 1 -- 172.21.15.31:0/1174064517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00ccc0 con 0x7fe8b4102a00 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8b1ffb640 1 -- 172.21.15.31:0/1174064517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ac00ce60 con 0x7fe8b4102a00 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8bad76640 1 -- 172.21.15.31:0/1174064517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b419efa0 con 0x7fe8b4102a00 2024-04-25T13:19:28.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8b1ffb640 1 -- 172.21.15.31:0/1174064517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac010430 con 0x7fe8b4102a00 2024-04-25T13:19:28.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.095+0000 7fe8bad76640 1 -- 172.21.15.31:0/1174064517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b419f4b0 con 0x7fe8b4102a00 2024-04-25T13:19:28.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.097+0000 7fe8bad76640 1 -- 172.21.15.31:0/1174064517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b4108250 con 0x7fe8b4102a00 2024-04-25T13:19:28.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.097+0000 7fe8b1ffb640 1 -- 172.21.15.31:0/1174064517 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fe8ac004f90 con 0x7fe8b4102a00 2024-04-25T13:19:28.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.098+0000 7fe8b1ffb640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe8a00a3d20 0x7fe8a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:28.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.098+0000 7fe8b8b12640 1 -- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe8a00a3d20 msgr2=0x7fe8a00a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/958678691 2024-04-25T13:19:28.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.098+0000 7fe8b1ffb640 1 -- 172.21.15.31:0/1174064517 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fe8ac0d0780 con 0x7fe8b4102a00 2024-04-25T13:19:28.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.098+0000 7fe8b8b12640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe8a00a3d20 0x7fe8a00a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:19:28.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.103+0000 7fe8b1ffb640 1 -- 172.21.15.31:0/1174064517 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8ac09d140 con 0x7fe8b4102a00 2024-04-25T13:19:28.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:27 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1251873866' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:19:28.219 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: ignoring --setuser ceph since I am not root 2024-04-25T13:19:28.220 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: ignoring --setgroup ceph since I am not root 2024-04-25T13:19:28.220 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-mgr[34920]: -- 172.21.15.136:0/2880451360 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a15bb05a0 con 0x555a15b84800 2024-04-25T13:19:28.220 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:28.143+0000 7f622a3fd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:19:28.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:27 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1251873866' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:19:28.234 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: ignoring --setuser ceph since I am not root 2024-04-25T13:19:28.234 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: ignoring --setgroup ceph since I am not root 2024-04-25T13:19:28.234 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:28.154+0000 7f580e129200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:19:28.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.294+0000 7fe8bad76640 1 -- 172.21.15.31:0/1174064517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fe8b4062b50 con 0x7fe8b4102a00 2024-04-25T13:19:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.299+0000 7fe8b8b12640 1 -- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe8a00a3d20 msgr2=0x7fe8a00a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/958678691 2024-04-25T13:19:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.299+0000 7fe8b8b12640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe8a00a3d20 0x7fe8a00a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:19:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.299+0000 7fe8b1ffb640 1 -- 172.21.15.31:0/1174064517 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7fe8ac09ca40 con 0x7fe8b4102a00 2024-04-25T13:19:28.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 -- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe8a00a3d20 msgr2=0x7fe8a00a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:19:28.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe8a00a3d20 0x7fe8a00a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 -- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b4102a00 msgr2=0x7fe8b419e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:28.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b4102a00 0x7fe8b419e640 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e970 tx=0x7fe8ac00ed70 comp rx=0 tx=0).stop 2024-04-25T13:19:28.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 -- 172.21.15.31:0/1174064517 shutdown_connections 2024-04-25T13:19:28.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fe8a00a3d20 0x7fe8a00a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8b4102a00 0x7fe8b419e640 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b4102040 0x7fe8b41975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 --2- 172.21.15.31:0/1174064517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8b4100e60 0x7fe8b4197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 -- 172.21.15.31:0/1174064517 >> 172.21.15.31:0/1174064517 conn(0x7fe8b40fc420 msgr2=0x7fe8b4105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:28.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 -- 172.21.15.31:0/1174064517 shutdown_connections 2024-04-25T13:19:28.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.302+0000 7fe8bad76640 1 -- 172.21.15.31:0/1174064517 wait complete. 2024-04-25T13:19:28.313 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T13:19:28.341 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:27 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1251873866' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:19:28.342 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: ignoring --setuser ceph since I am not root 2024-04-25T13:19:28.342 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: ignoring --setgroup ceph since I am not root 2024-04-25T13:19:28.342 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-mgr[34527]: -- 172.21.15.66:0/3296419488 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563af67345a0 con 0x563af6708c00 2024-04-25T13:19:28.342 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:28.149+0000 7efffb058200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:19:28.342 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:28.339+0000 7efffb058200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:19:28.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.416+0000 7f3f050e1640 1 -- 172.21.15.31:0/623830947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00102040 msgr2=0x7f3f001024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:28.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.416+0000 7f3f050e1640 1 --2- 172.21.15.31:0/623830947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00102040 0x7f3f001024c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009900 tx=0x7f3ef402a410 comp rx=0 tx=0).stop 2024-04-25T13:19:28.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.416+0000 7f3f050e1640 1 -- 172.21.15.31:0/623830947 shutdown_connections 2024-04-25T13:19:28.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.416+0000 7f3f050e1640 1 --2- 172.21.15.31:0/623830947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f00102a00 0x7f3f00109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.416+0000 7f3f050e1640 1 --2- 172.21.15.31:0/623830947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00102040 0x7f3f001024c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.416+0000 7f3f050e1640 1 --2- 172.21.15.31:0/623830947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f00100e60 0x7f3f00101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.416+0000 7f3f050e1640 1 -- 172.21.15.31:0/623830947 >> 172.21.15.31:0/623830947 conn(0x7f3f000fc420 msgr2=0x7f3f000fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:28.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.416+0000 7f3f050e1640 1 -- 172.21.15.31:0/623830947 shutdown_connections 2024-04-25T13:19:28.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.416+0000 7f3f050e1640 1 -- 172.21.15.31:0/623830947 wait complete. 2024-04-25T13:19:28.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.417+0000 7f3f050e1640 1 Processor -- start 2024-04-25T13:19:28.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.417+0000 7f3f050e1640 1 -- start start 2024-04-25T13:19:28.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3f050e1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f00100e60 0x7f3f0006a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:28.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3f050e1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00102040 0x7f3f0019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:28.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3f050e1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f00102a00 0x7f3f0019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3f050e1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0010ca10 con 0x7f3f00102040 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3f050e1640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0010c890 con 0x7f3f00100e60 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3f050e1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0010cb90 con 0x7f3f00102a00 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3efe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00102040 0x7f3f0019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3efed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f00100e60 0x7f3f0006a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3efe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00102040 0x7f3f0019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34836/0 (socket says 172.21.15.31:34836) 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3efe59c640 1 -- 172.21.15.31:0/1890426654 learned_addr learned my addr 172.21.15.31:0/1890426654 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3eff59e640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f00102a00 0x7f3f0019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3efe59c640 1 -- 172.21.15.31:0/1890426654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f00102a00 msgr2=0x7f3f0019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3efe59c640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f00102a00 0x7f3f0019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3efe59c640 1 -- 172.21.15.31:0/1890426654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f00100e60 msgr2=0x7f3f0006a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3efe59c640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f00100e60 0x7f3f0006a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3efe59c640 1 -- 172.21.15.31:0/1890426654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0019fd90 con 0x7f3f00102040 2024-04-25T13:19:28.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3efe59c640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00102040 0x7f3f0019ce90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4005940 tx=0x7f3ef40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3ee7fff640 1 -- 172.21.15.31:0/1890426654 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef403f030 con 0x7f3f00102040 2024-04-25T13:19:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3ee7fff640 1 -- 172.21.15.31:0/1890426654 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef4003cb0 con 0x7f3f00102040 2024-04-25T13:19:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3f050e1640 1 -- 172.21.15.31:0/1890426654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f001a0020 con 0x7f3f00102040 2024-04-25T13:19:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.418+0000 7f3ee7fff640 1 -- 172.21.15.31:0/1890426654 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4035070 con 0x7f3f00102040 2024-04-25T13:19:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.419+0000 7f3f050e1640 1 -- 172.21.15.31:0/1890426654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f001a0560 con 0x7f3f00102040 2024-04-25T13:19:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.419+0000 7f3f050e1640 1 -- 172.21.15.31:0/1890426654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f00108280 con 0x7f3f00102040 2024-04-25T13:19:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.421+0000 7f3ee7fff640 1 -- 172.21.15.31:0/1890426654 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f3ef4042030 con 0x7f3f00102040 2024-04-25T13:19:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.421+0000 7f3ee7fff640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ee80a3d20 0x7f3ee80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.421+0000 7f3efed9d640 1 -- 172.21.15.31:0/1890426654 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ee80a3d20 msgr2=0x7f3ee80a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/958678691 2024-04-25T13:19:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.421+0000 7f3ee7fff640 1 -- 172.21.15.31:0/1890426654 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f3ef40f0e30 con 0x7f3f00102040 2024-04-25T13:19:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.421+0000 7f3efed9d640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ee80a3d20 0x7f3ee80a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:19:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.425+0000 7f3ee7fff640 1 -- 172.21.15.31:0/1890426654 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ef4003e60 con 0x7f3f00102040 2024-04-25T13:19:28.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.612+0000 7f3f050e1640 1 -- 172.21.15.31:0/1890426654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f3f00062b50 con 0x7f3f00102040 2024-04-25T13:19:28.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.617+0000 7f3ee7fff640 1 -- 172.21.15.31:0/1890426654 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f3ef40bd1a0 con 0x7f3f00102040 2024-04-25T13:19:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 -- 172.21.15.31:0/1890426654 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ee80a3d20 msgr2=0x7f3ee80a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:19:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ee80a3d20 0x7f3ee80a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 -- 172.21.15.31:0/1890426654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00102040 msgr2=0x7f3f0019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00102040 0x7f3f0019ce90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4005940 tx=0x7f3ef40042f0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 -- 172.21.15.31:0/1890426654 shutdown_connections 2024-04-25T13:19:28.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7f3ee80a3d20 0x7f3ee80a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f00102a00 0x7f3f0019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00102040 0x7f3f0019ce90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 --2- 172.21.15.31:0/1890426654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f00100e60 0x7f3f0006a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 -- 172.21.15.31:0/1890426654 >> 172.21.15.31:0/1890426654 conn(0x7f3f000fc420 msgr2=0x7f3f00105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 -- 172.21.15.31:0/1890426654 shutdown_connections 2024-04-25T13:19:28.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:28.620+0000 7f3f050e1640 1 -- 172.21.15.31:0/1890426654 wait complete. 2024-04-25T13:19:28.630 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:28.349+0000 7f580e129200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:19:28.630 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:28.462+0000 7f580e129200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:19:28.632 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T13:19:28.672 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T13:19:28.673 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T13:19:28.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:19:28.673 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-25T13:19:28.674 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:28.445+0000 7efffb058200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:19:28.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:19:28.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-04-25T13:19:28.718 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:28.328+0000 7f622a3fd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:19:28.719 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:28.431+0000 7f622a3fd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:19:28.756 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2024-04-25T13:19:28.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:19:28.758 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-25T13:19:28.758 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T13:19:28.758 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:19:28.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T13:19:28.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:28.788+0000 7f580e129200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:19:28.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:28.880+0000 7f580e129200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:19:29.053 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:19:29.158 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:28.763+0000 7efffb058200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:19:29.158 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:28.850+0000 7efffb058200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:19:29.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1251873866' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:19:29.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-mon[33137]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T13:19:29.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1174064517' entity='client.admin' 2024-04-25T13:19:29.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:28 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1890426654' entity='client.admin' 2024-04-25T13:19:29.219 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:28.738+0000 7f622a3fd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:19:29.219 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:28.824+0000 7f622a3fd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:19:29.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1251873866' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:19:29.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-mon[33038]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T13:19:29.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1174064517' entity='client.admin' 2024-04-25T13:19:29.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:28 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1890426654' entity='client.admin' 2024-04-25T13:19:29.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:28 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1251873866' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:19:29.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:28 smithi031 ceph-mon[29457]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T13:19:29.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:28 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1174064517' entity='client.admin' 2024-04-25T13:19:29.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:28 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1890426654' entity='client.admin' 2024-04-25T13:19:30.349 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:30.006+0000 7f622a3fd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:19:30.349 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:30.177+0000 7f622a3fd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:19:30.408 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:30.091+0000 7efffb058200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:19:30.408 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:30.267+0000 7efffb058200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:19:30.465 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:30.160+0000 7f580e129200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:19:30.465 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:30.344+0000 7f580e129200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:19:30.636 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:30.348+0000 7f622a3fd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:19:30.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.708+0000 7ff258383700 1 -- 172.21.15.31:0/394446515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff250101930 msgr2=0x7ff250101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:30.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.708+0000 7ff258383700 1 --2- 172.21.15.31:0/394446515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff250101930 0x7ff250101db0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009990 tx=0x7ff24c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.708+0000 7ff258383700 1 -- 172.21.15.31:0/394446515 shutdown_connections 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.708+0000 7ff258383700 1 --2- 172.21.15.31:0/394446515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501022f0 0x7ff25010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.708+0000 7ff258383700 1 --2- 172.21.15.31:0/394446515 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff250107980 0x7ff250107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.708+0000 7ff258383700 1 --2- 172.21.15.31:0/394446515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff250101930 0x7ff250101db0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.708+0000 7ff258383700 1 -- 172.21.15.31:0/394446515 >> 172.21.15.31:0/394446515 conn(0x7ff2500788b0 msgr2=0x7ff250078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.708+0000 7ff258383700 1 -- 172.21.15.31:0/394446515 shutdown_connections 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.708+0000 7ff258383700 1 -- 172.21.15.31:0/394446515 wait complete. 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff258383700 1 Processor -- start 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff258383700 1 -- start start 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff258383700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff250101930 0x7ff2501034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff258383700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501022f0 0x7ff250103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff258383700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff250107980 0x7ff250103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff258383700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff250111ae0 con 0x7ff250101930 2024-04-25T13:19:30.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff258383700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff250111960 con 0x7ff250107980 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff258383700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff250111c60 con 0x7ff2501022f0 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff25611f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff250101930 0x7ff2501034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff25611f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff250101930 0x7ff2501034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34850/0 (socket says 172.21.15.31:34850) 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff25611f700 1 -- 172.21.15.31:0/3219571458 learned_addr learned my addr 172.21.15.31:0/3219571458 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.709+0000 7ff25591e700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501022f0 0x7ff250103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff256920700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff250107980 0x7ff250103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff25611f700 1 -- 172.21.15.31:0/3219571458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501022f0 msgr2=0x7ff250103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff25611f700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501022f0 0x7ff250103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff25611f700 1 -- 172.21.15.31:0/3219571458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff250107980 msgr2=0x7ff250103f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff25611f700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff250107980 0x7ff250103f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff25611f700 1 -- 172.21.15.31:0/3219571458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2501a76a0 con 0x7ff250101930 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff25611f700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff250101930 0x7ff2501034e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e9a0 tx=0x7ff24400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff2437fe700 1 -- 172.21.15.31:0/3219571458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24400ca10 con 0x7ff250101930 2024-04-25T13:19:30.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff258383700 1 -- 172.21.15.31:0/3219571458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2501a7990 con 0x7ff250101930 2024-04-25T13:19:30.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff2437fe700 1 -- 172.21.15.31:0/3219571458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24400cbb0 con 0x7ff250101930 2024-04-25T13:19:30.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff2437fe700 1 -- 172.21.15.31:0/3219571458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244018680 con 0x7ff250101930 2024-04-25T13:19:30.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.710+0000 7ff258383700 1 -- 172.21.15.31:0/3219571458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2501a7ef0 con 0x7ff250101930 2024-04-25T13:19:30.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.711+0000 7ff258383700 1 -- 172.21.15.31:0/3219571458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff250069730 con 0x7ff250101930 2024-04-25T13:19:30.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.713+0000 7ff2437fe700 1 -- 172.21.15.31:0/3219571458 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7ff244010430 con 0x7ff250101930 2024-04-25T13:19:30.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.714+0000 7ff2437fe700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff23c0a3c90 0x7ff23c0a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:30.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.714+0000 7ff25591e700 1 -- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff23c0a3c90 msgr2=0x7ff23c0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/958678691 2024-04-25T13:19:30.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.714+0000 7ff2437fe700 1 -- 172.21.15.31:0/3219571458 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7ff2440d0910 con 0x7ff250101930 2024-04-25T13:19:30.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.714+0000 7ff25591e700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff23c0a3c90 0x7ff23c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:19:30.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.717+0000 7ff2437fe700 1 -- 172.21.15.31:0/3219571458 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff24409d270 con 0x7ff250101930 2024-04-25T13:19:30.717 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:30.530+0000 7f580e129200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:19:30.742 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:30.442+0000 7efffb058200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:19:30.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.915+0000 7ff25591e700 1 -- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff23c0a3c90 msgr2=0x7ff23c0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/958678691 2024-04-25T13:19:30.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.915+0000 7ff25591e700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff23c0a3c90 0x7ff23c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:19:30.934 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:30.636+0000 7f622a3fd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:19:30.934 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:30.729+0000 7f622a3fd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:19:30.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.972+0000 7ff258383700 1 -- 172.21.15.31:0/3219571458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff25002df80 con 0x7ff250101930 2024-04-25T13:19:30.973 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:30.840+0000 7f580e129200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:19:30.973 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:30.937+0000 7f580e129200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:19:30.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.973+0000 7ff2437fe700 1 -- 172.21.15.31:0/3219571458 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7ff24409cb70 con 0x7ff250101930 2024-04-25T13:19:30.973 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:19:30.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 -- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff23c0a3c90 msgr2=0x7ff23c0a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:19:30.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff23c0a3c90 0x7ff23c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 -- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff250101930 msgr2=0x7ff2501034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:30.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff250101930 0x7ff2501034e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e9a0 tx=0x7ff24400ecb0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 -- 172.21.15.31:0/3219571458 shutdown_connections 2024-04-25T13:19:30.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff250107980 0x7ff250103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff250101930 0x7ff2501034e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7ff23c0a3c90 0x7ff23c0a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 --2- 172.21.15.31:0/3219571458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2501022f0 0x7ff250103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 -- 172.21.15.31:0/3219571458 >> 172.21.15.31:0/3219571458 conn(0x7ff2500788b0 msgr2=0x7ff25006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:30.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 -- 172.21.15.31:0/3219571458 shutdown_connections 2024-04-25T13:19:30.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:30.975+0000 7ff258383700 1 -- 172.21.15.31:0/3219571458 wait complete. 2024-04-25T13:19:31.024 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:30.740+0000 7efffb058200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:19:31.025 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:30.836+0000 7efffb058200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:19:31.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:31 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3219571458' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:19:31.219 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:30.933+0000 7f622a3fd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:19:31.219 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:31 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:31.129+0000 7f622a3fd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:19:31.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:31 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3219571458' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:19:31.233 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:31 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:31.152+0000 7f580e129200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:19:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:31 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3219571458' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:19:31.408 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:31 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:31.046+0000 7efffb058200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:19:31.408 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:31 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:31.246+0000 7efffb058200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:19:31.691 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:31 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:31.358+0000 7f580e129200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:19:31.720 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:19:31.720 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:19:31.720 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-04-25T13:19:31.720 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:19:31.720 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:19:31.720 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:19:31.720 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:19:31.720 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:19:31.720 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:19:31.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:19:31.721 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-04-25T13:19:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:19:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:19:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:19:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:19:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-04-25T13:19:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:19:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41998sec preferred_lft 41998sec 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:19:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-04-25T13:19:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-04-25T13:19:31.739 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T13:19:31.739 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:19:31.739 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T13:19:31.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-04-25T13:19:31.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-04-25T13:19:31.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-04-25T13:19:31.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T13:19:31.740 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T13:19:31.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:19:31.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:19:31.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:19:31.929 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:19:31.955 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-04-25T13:19:31.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:19:31.957 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-04-25T13:19:32.022 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-04-25T13:19:32.022 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:19:32.022 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T13:19:32.022 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T13:19:32.022 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:19:32.023 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:19:32.023 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T13:19:32.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T13:19:32.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T13:19:32.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T13:19:32.023 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T13:19:32.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:19:32.157 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:31 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:31.869+0000 7efffb058200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:19:32.160 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:31 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:31.731+0000 7f622a3fd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:19:32.210 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:32 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:32.000+0000 7f580e129200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:19:32.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:19:32.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:19:32.293 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-25T13:19:32.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:19:32.308 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-04-25T13:19:32.363 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:19:32.363 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T13:19:32.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:19:32.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T13:19:32.363 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T13:19:32.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:19:32.460 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:32 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:32.460+0000 7f580e129200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:19:32.461 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:32.159+0000 7f622a3fd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:19:32.461 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:32.256+0000 7f622a3fd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:19:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:19:32.482 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T13:19:32.482 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:19:32.482 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T13:19:32.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T13:19:32.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T13:19:32.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T13:19:32.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T13:19:32.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T13:19:32.483 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T13:19:32.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:19:32.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:19:32.612 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:19:32.612 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-25T13:19:32.612 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-04-25T13:19:32.612 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T13:19:32.612 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-25T13:19:32.616 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:32 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:32.306+0000 7efffb058200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:19:32.616 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:32 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:32.406+0000 7efffb058200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:19:32.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:19:32.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:19:32.691 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:19:32.691 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:19:32.706 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:19:32.706 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:19:32.706 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:19:32.718 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:32.460+0000 7f622a3fd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:19:32.734 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:32 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:32.571+0000 7f580e129200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:19:32.748 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T13:19:32.748 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T13:19:32.748 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/4313c4b3-3c68-4ee3-ad7e-c53ba53f1895 2024-04-25T13:19:32.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:19:32.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/4313c4b3-3c68-4ee3-ad7e-c53ba53f1895 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T13:19:32.851 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T13:19:32.851 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-25T13:19:32.852 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-04-25T13:19:32.908 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:32 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:32.614+0000 7efffb058200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:19:32.971 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T13:19:32.971 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-25T13:19:32.971 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-04-25T13:19:32.971 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.66:3300,172.21.15.136:3300" 2024-04-25T13:19:32.971 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:3300,172.21.15.66:3300,172.21.15.136:3300:/volumes/qa/sv_1/4313c4b3-3c68-4ee3-ad7e-c53ba53f1895 2024-04-25T13:19:32.972 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=e4b023f4-0304-11ef-bc93-c7b262605968" will pass to kernel 2024-04-25T13:19:32.972 INFO:teuthology.orchestra.run.smithi031.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T13:19:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: You just mounted a file system that supports labels which does not 2024-04-25T13:19:32.972 INFO:teuthology.orchestra.run.smithi031.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T13:19:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T13:19:32.973 INFO:teuthology.orchestra.run.smithi031.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T13:19:32.974 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T13:19:32.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:19:32.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:19:33.002 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:19:33.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:19:33.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:19:33.100 INFO:teuthology.orchestra.run.smithi031.stdout:client24911 2024-04-25T13:19:33.101 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:19:33.101 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:19:33.117 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:19:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:19:33.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:19:33.216 INFO:teuthology.orchestra.run.smithi031.stdout:client24911 2024-04-25T13:19:33.217 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:19:33.233 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:32 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:32.791+0000 7f580e129200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:19:33.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:19:33.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:19:33.273 INFO:teuthology.orchestra.run.smithi031.stdout:client24911 2024-04-25T13:19:33.274 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e4b023f4-0304-11ef-bc93-c7b262605968.client24911/status') 2024-04-25T13:19:33.324 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:32.987+0000 7f622a3fd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:19:33.324 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:33.080+0000 7f622a3fd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:19:33.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:19:33.373 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.24911 (3)172.21.15.31:0/3021239815 2024-04-25T13:19:33.373 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2024-04-25T13:19:33.373 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2024-04-25T13:19:33.373 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2024-04-25T13:19:33.373 INFO:teuthology.orchestra.run.smithi031.stderr:71 bytes copied, 2.5818e-05 s, 2.8 MB/s 2024-04-25T13:19:33.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:19:33.381 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T13:19:33.396 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T13:19:33.404 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:19:33.404 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T13:19:33.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T13:19:33.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.509+0000 7fa87cacf640 1 -- 172.21.15.31:0/1770803063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa878101fc0 msgr2=0x7fa8781023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:33.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.509+0000 7fa87cacf640 1 --2- 172.21.15.31:0/1770803063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa878101fc0 0x7fa8781023e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa868009900 tx=0x7fa86802a410 comp rx=0 tx=0).stop 2024-04-25T13:19:33.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.509+0000 7fa87cacf640 1 -- 172.21.15.31:0/1770803063 shutdown_connections 2024-04-25T13:19:33.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.509+0000 7fa87cacf640 1 --2- 172.21.15.31:0/1770803063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa878103b70 0x7fa87810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:33.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.509+0000 7fa87cacf640 1 --2- 172.21.15.31:0/1770803063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8781031b0 0x7fa878103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:33.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.509+0000 7fa87cacf640 1 --2- 172.21.15.31:0/1770803063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa878101fc0 0x7fa8781023e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:33.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.509+0000 7fa87cacf640 1 -- 172.21.15.31:0/1770803063 >> 172.21.15.31:0/1770803063 conn(0x7fa8780fd530 msgr2=0x7fa8780ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:33.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.509+0000 7fa87cacf640 1 -- 172.21.15.31:0/1770803063 shutdown_connections 2024-04-25T13:19:33.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.509+0000 7fa87cacf640 1 -- 172.21.15.31:0/1770803063 wait complete. 2024-04-25T13:19:33.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.510+0000 7fa87cacf640 1 Processor -- start 2024-04-25T13:19:33.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.510+0000 7fa87cacf640 1 -- start start 2024-04-25T13:19:33.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.510+0000 7fa87cacf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa878101fc0 0x7fa878198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.510+0000 7fa87cacf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8781031b0 0x7fa878198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.510+0000 7fa87cacf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa878103b70 0x7fa87819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.510+0000 7fa87cacf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa87810db00 con 0x7fa878101fc0 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.510+0000 7fa87cacf640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa87810d980 con 0x7fa878103b70 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.510+0000 7fa87cacf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa87810dc80 con 0x7fa8781031b0 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa87659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa878101fc0 0x7fa878198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa876d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa878103b70 0x7fa87819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa87659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa878101fc0 0x7fa878198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40062/0 (socket says 172.21.15.31:40062) 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa87659c640 1 -- 172.21.15.31:0/3115689712 learned_addr learned my addr 172.21.15.31:0/3115689712 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa86dd9b640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8781031b0 0x7fa878198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa87659c640 1 -- 172.21.15.31:0/3115689712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8781031b0 msgr2=0x7fa878198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa87659c640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8781031b0 0x7fa878198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa87659c640 1 -- 172.21.15.31:0/3115689712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa878103b70 msgr2=0x7fa87819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:33.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa87659c640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa878103b70 0x7fa87819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:33.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa87659c640 1 -- 172.21.15.31:0/3115689712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87819fd60 con 0x7fa878101fc0 2024-04-25T13:19:33.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa876d9d640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa878103b70 0x7fa87819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:19:33.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa87659c640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa878101fc0 0x7fa878198140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa868006010 tx=0x7fa8680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:33.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86803f030 con 0x7fa878101fc0 2024-04-25T13:19:33.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa87cacf640 1 -- 172.21.15.31:0/3115689712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87819fff0 con 0x7fa878101fc0 2024-04-25T13:19:33.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8680318f0 con 0x7fa878101fc0 2024-04-25T13:19:33.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868035070 con 0x7fa878101fc0 2024-04-25T13:19:33.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.511+0000 7fa87cacf640 1 -- 172.21.15.31:0/3115689712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8781a0450 con 0x7fa878101fc0 2024-04-25T13:19:33.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.512+0000 7fa87cacf640 1 -- 172.21.15.31:0/3115689712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa878109360 con 0x7fa878101fc0 2024-04-25T13:19:33.515 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:33.166+0000 7efffb058200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:19:33.515 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:33.261+0000 7efffb058200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:19:33.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.514+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fa868042030 con 0x7fa878101fc0 2024-04-25T13:19:33.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.514+0000 7fa86ffff640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa8640a3ca0 0x7fa8640a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.514+0000 7fa86dd9b640 1 -- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa8640a3ca0 msgr2=0x7fa8640a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/958678691 2024-04-25T13:19:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.514+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fa8680f0cc0 con 0x7fa878101fc0 2024-04-25T13:19:33.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.514+0000 7fa86dd9b640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa8640a3ca0 0x7fa8640a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:19:33.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.518+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8680bd030 con 0x7fa878101fc0 2024-04-25T13:19:33.671 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:33.324+0000 7f622a3fd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:19:33.671 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:33.457+0000 7f622a3fd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:19:33.671 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:33.554+0000 7f622a3fd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:19:33.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.715+0000 7fa86dd9b640 1 -- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa8640a3ca0 msgr2=0x7fa8640a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/958678691 2024-04-25T13:19:33.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.715+0000 7fa86dd9b640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa8640a3ca0 0x7fa8640a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:19:33.721 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:33 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:33.356+0000 7f580e129200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:19:33.721 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:33 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:33.458+0000 7f580e129200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:19:33.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:33.729+0000 7fa87cacf640 1 -- 172.21.15.31:0/3115689712 --> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fa878062390 con 0x7fa8640a3ca0 2024-04-25T13:19:33.869 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:33.512+0000 7efffb058200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:19:33.869 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:33.650+0000 7efffb058200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:19:33.869 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:33.748+0000 7efffb058200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:19:33.983 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:33 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:33.721+0000 7f580e129200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:19:33.984 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:33 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:33.862+0000 7f580e129200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:19:33.984 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:33 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:33.963+0000 7f580e129200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:19:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:34.116+0000 7fa86dd9b640 1 -- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa8640a3ca0 msgr2=0x7fa8640a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/958678691 2024-04-25T13:19:34.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:34.116+0000 7fa86dd9b640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa8640a3ca0 0x7fa8640a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:19:34.157 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:33.867+0000 7efffb058200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:19:34.399 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:34 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:34.085+0000 7f580e129200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:19:34.469 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:33.670+0000 7f622a3fd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:19:34.469 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:33.976+0000 7f622a3fd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:19:34.658 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:34 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:34.178+0000 7efffb058200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:19:34.733 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:34 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:34.399+0000 7f580e129200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:19:34.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:34.863+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7fa8680b96a0 con 0x7fa878101fc0 2024-04-25T13:19:34.865 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:34 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:34.581+0000 7f622a3fd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:19:34.866 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:19:34 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z[34916]: 2024-04-25T13:19:34.800+0000 7f622a3fd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:19:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:34.917+0000 7fa86dd9b640 1 -- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa8640a3ca0 msgr2=0x7fa8640a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/958678691 2024-04-25T13:19:34.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:34.917+0000 7fa86dd9b640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa8640a3ca0 0x7fa8640a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:19:35.158 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:34 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:34.804+0000 7efffb058200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:19:35.158 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y[34523]: 2024-04-25T13:19:35.029+0000 7efffb058200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:19:35.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:34 smithi066 ceph-mon[33137]: Standby manager daemon z restarted 2024-04-25T13:19:35.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:34 smithi066 ceph-mon[33137]: Standby manager daemon z started 2024-04-25T13:19:35.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:34 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:19:35.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:34 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:19:35.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:34 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:19:35.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:34 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:19:35.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:34 smithi136 ceph-mon[33038]: Standby manager daemon z restarted 2024-04-25T13:19:35.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:34 smithi136 ceph-mon[33038]: Standby manager daemon z started 2024-04-25T13:19:35.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:34 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:19:35.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:34 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:19:35.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:34 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:19:35.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:34 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:19:35.233 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.038+0000 7f580e129200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:19:35.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:34 smithi031 ceph-mon[29457]: Standby manager daemon z restarted 2024-04-25T13:19:35.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:34 smithi031 ceph-mon[29457]: Standby manager daemon z started 2024-04-25T13:19:35.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:34 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:19:35.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:34 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:19:35.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:34 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:19:35.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:34 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.136:0/1445483446' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:19:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:35.280+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fa86803bb00 con 0x7fa878101fc0 2024-04-25T13:19:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:35.280+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa8640a3ca0 msgr2=0x7fa8640a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:19:35.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:35.280+0000 7fa86ffff640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/958678691,v1:172.21.15.31:6801/958678691] conn(0x7fa8640a3ca0 0x7fa8640a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:35.527 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.268+0000 7f580e129200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:19:35.868 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.526+0000 7f576ca5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:19:35.868 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.526+0000 7f576ca5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:19:35.868 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.526+0000 7f576ca5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:19:35.868 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.526+0000 7f576ca5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:19:35.869 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.526+0000 7f576ca5b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:19:35.869 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.526+0000 7f576f260700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:19:35.869 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.526+0000 7f576f260700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:19:35.869 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.526+0000 7f576f260700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:19:35.869 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.526+0000 7f576f260700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:19:35.869 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:19:35.526+0000 7f576f260700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:19:36.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: Standby manager daemon y restarted 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: Standby manager daemon y started 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: Active manager daemon x restarted 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: Activating manager daemon x 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: mgrmap e21: x(active, starting, since 0.0106222s), standbys: y, z 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:36.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:19:36.160 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:19:36.161 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:19:36.162 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:19:36.162 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:19:36.162 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: Manager daemon x is now available 2024-04-25T13:19:36.162 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:36.162 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:19:36.162 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:19:36.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-25T13:19:36.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: Standby manager daemon y restarted 2024-04-25T13:19:36.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: Standby manager daemon y started 2024-04-25T13:19:36.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: Active manager daemon x restarted 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: Activating manager daemon x 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: mgrmap e21: x(active, starting, since 0.0106222s), standbys: y, z 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:19:36.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:36.221 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: Manager daemon x is now available 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:36.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:19:36.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:19:36.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-25T13:19:36.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: Standby manager daemon y restarted 2024-04-25T13:19:36.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: Standby manager daemon y started 2024-04-25T13:19:36.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.? 172.21.15.66:0/2117733776' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: Active manager daemon x restarted 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: Activating manager daemon x 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: mgrmap e21: x(active, starting, since 0.0106222s), standbys: y, z 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:19:36.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:19:36.236 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: Manager daemon x is now available 2024-04-25T13:19:36.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:36.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:19:36.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:19:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.286+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 <== mon.0 v2:172.21.15.31:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fa8680ee7f0 con 0x7fa878101fc0 2024-04-25T13:19:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.287+0000 7fa86ffff640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fa864035890 0x7fa864035c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.287+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 --> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fa86802cea0 con 0x7fa864035890 2024-04-25T13:19:36.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.294+0000 7fa86dd9b640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fa864035890 0x7fa864035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:36.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.297+0000 7fa86dd9b640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fa864035890 0x7fa864035c70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa878199650 tx=0x7fa86000b410 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:36.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.374+0000 7fa86ffff640 1 -- 172.21.15.31:0/3115689712 <== mgr.24922 v2:172.21.15.31:6800/332946768 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa86802cea0 con 0x7fa864035890 2024-04-25T13:19:36.375 INFO:teuthology.orchestra.run.smithi031.stdout:Schedule set for path / 2024-04-25T13:19:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.377+0000 7fa87cacf640 1 -- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fa864035890 msgr2=0x7fa864035c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.377+0000 7fa87cacf640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fa864035890 0x7fa864035c70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa878199650 tx=0x7fa86000b410 comp rx=0 tx=0).stop 2024-04-25T13:19:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.377+0000 7fa87cacf640 1 -- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa878101fc0 msgr2=0x7fa878198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.377+0000 7fa87cacf640 1 --2- 172.21.15.31:0/3115689712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa878101fc0 0x7fa878198140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa868006010 tx=0x7fa8680042f0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.377+0000 7fa876d9d640 1 -- 172.21.15.31:0/3115689712 reap_dead start 2024-04-25T13:19:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.377+0000 7fa87cacf640 1 -- 172.21.15.31:0/3115689712 shutdown_connections 2024-04-25T13:19:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.377+0000 7fa87cacf640 1 -- 172.21.15.31:0/3115689712 >> 172.21.15.31:0/3115689712 conn(0x7fa8780fd530 msgr2=0x7fa878106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.377+0000 7fa87cacf640 1 -- 172.21.15.31:0/3115689712 shutdown_connections 2024-04-25T13:19:36.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.377+0000 7fa87cacf640 1 -- 172.21.15.31:0/3115689712 wait complete. 2024-04-25T13:19:36.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T13:19:36.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.494+0000 7ff62bf37640 1 -- 172.21.15.31:0/787271074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff624100290 msgr2=0x7ff6241006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:36.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.494+0000 7ff62bf37640 1 --2- 172.21.15.31:0/787271074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff624100290 0x7ff6241006b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff620009930 tx=0x7ff62002a420 comp rx=0 tx=0).stop 2024-04-25T13:19:36.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.494+0000 7ff62bf37640 1 -- 172.21.15.31:0/787271074 shutdown_connections 2024-04-25T13:19:36.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.494+0000 7ff62bf37640 1 --2- 172.21.15.31:0/787271074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff624103c20 0x7ff62410a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.494+0000 7ff62bf37640 1 --2- 172.21.15.31:0/787271074 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624103260 0x7ff6241036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.494+0000 7ff62bf37640 1 --2- 172.21.15.31:0/787271074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff624100290 0x7ff6241006b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.494+0000 7ff62bf37640 1 -- 172.21.15.31:0/787271074 >> 172.21.15.31:0/787271074 conn(0x7ff6240fbbc0 msgr2=0x7ff6240fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:36.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.494+0000 7ff62bf37640 1 -- 172.21.15.31:0/787271074 shutdown_connections 2024-04-25T13:19:36.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.494+0000 7ff62bf37640 1 -- 172.21.15.31:0/787271074 wait complete. 2024-04-25T13:19:36.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff62bf37640 1 Processor -- start 2024-04-25T13:19:36.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff62bf37640 1 -- start start 2024-04-25T13:19:36.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff62bf37640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff624100290 0x7ff624198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:36.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff62bf37640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff624103260 0x7ff6241986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:36.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff62bf37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624103c20 0x7ff62419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:36.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff62bf37640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff62410db30 con 0x7ff624103260 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff62bf37640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff62410d9b0 con 0x7ff624103c20 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff62bf37640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff62410dcb0 con 0x7ff624100290 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff6294d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff624103260 0x7ff6241986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff6294d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff624103260 0x7ff6241986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40204/0 (socket says 172.21.15.31:40204) 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff6294d2640 1 -- 172.21.15.31:0/1603253820 learned_addr learned my addr 172.21.15.31:0/1603253820 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff629cd3640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff624100290 0x7ff624198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.495+0000 7ff62a4d4640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624103c20 0x7ff62419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.496+0000 7ff6294d2640 1 -- 172.21.15.31:0/1603253820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff624100290 msgr2=0x7ff624198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.496+0000 7ff6294d2640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff624100290 0x7ff624198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.496+0000 7ff6294d2640 1 -- 172.21.15.31:0/1603253820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624103c20 msgr2=0x7ff62419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.496+0000 7ff6294d2640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624103c20 0x7ff62419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.496+0000 7ff6294d2640 1 -- 172.21.15.31:0/1603253820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62419fd80 con 0x7ff624103260 2024-04-25T13:19:36.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.496+0000 7ff6294d2640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff624103260 0x7ff6241986a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff61400d950 tx=0x7ff61400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:36.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.496+0000 7ff61affd640 1 -- 172.21.15.31:0/1603253820 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614017070 con 0x7ff624103260 2024-04-25T13:19:36.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.496+0000 7ff61affd640 1 -- 172.21.15.31:0/1603253820 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff614009c30 con 0x7ff624103260 2024-04-25T13:19:36.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.496+0000 7ff62bf37640 1 -- 172.21.15.31:0/1603253820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6241a0070 con 0x7ff624103260 2024-04-25T13:19:36.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.496+0000 7ff61affd640 1 -- 172.21.15.31:0/1603253820 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61400f620 con 0x7ff624103260 2024-04-25T13:19:36.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.496+0000 7ff62bf37640 1 -- 172.21.15.31:0/1603253820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6241a05c0 con 0x7ff624103260 2024-04-25T13:19:36.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.497+0000 7ff62bf37640 1 -- 172.21.15.31:0/1603253820 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff624109390 con 0x7ff624103260 2024-04-25T13:19:36.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.498+0000 7ff61affd640 1 -- 172.21.15.31:0/1603253820 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7ff614010460 con 0x7ff624103260 2024-04-25T13:19:36.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.499+0000 7ff61affd640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7ff6100a3a60 0x7ff6100a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:36.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.499+0000 7ff629cd3640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7ff6100a3a60 0x7ff6100a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:36.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.499+0000 7ff61affd640 1 -- 172.21.15.31:0/1603253820 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8527+0+0 (secure 0 0 0) 0x7ff6140cfd90 con 0x7ff624103260 2024-04-25T13:19:36.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.499+0000 7ff629cd3640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7ff6100a3a60 0x7ff6100a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff620009900 tx=0x7ff6200314a0 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:36.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.503+0000 7ff61affd640 1 -- 172.21.15.31:0/1603253820 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff61409c900 con 0x7ff624103260 2024-04-25T13:19:36.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.723+0000 7ff62bf37640 1 -- 172.21.15.31:0/1603253820 --> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7ff6240624f0 con 0x7ff6100a3a60 2024-04-25T13:19:36.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.784+0000 7ff61affd640 1 -- 172.21.15.31:0/1603253820 <== mgr.24922 v2:172.21.15.31:6800/332946768 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7ff6240624f0 con 0x7ff6100a3a60 2024-04-25T13:19:36.784 INFO:teuthology.orchestra.run.smithi031.stdout:Retention added to path / 2024-04-25T13:19:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 -- 172.21.15.31:0/1603253820 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7ff6100a3a60 msgr2=0x7ff6100a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:36.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7ff6100a3a60 0x7ff6100a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff620009900 tx=0x7ff6200314a0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 -- 172.21.15.31:0/1603253820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff624103260 msgr2=0x7ff6241986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff624103260 0x7ff6241986a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff61400d950 tx=0x7ff61400de10 comp rx=0 tx=0).stop 2024-04-25T13:19:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 -- 172.21.15.31:0/1603253820 shutdown_connections 2024-04-25T13:19:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7ff6100a3a60 0x7ff6100a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff624103c20 0x7ff62419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff624103260 0x7ff6241986a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 --2- 172.21.15.31:0/1603253820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff624100290 0x7ff624198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 -- 172.21.15.31:0/1603253820 >> 172.21.15.31:0/1603253820 conn(0x7ff6240fbbc0 msgr2=0x7ff624106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 -- 172.21.15.31:0/1603253820 shutdown_connections 2024-04-25T13:19:36.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.786+0000 7ff62bf37640 1 -- 172.21.15.31:0/1603253820 wait complete. 2024-04-25T13:19:36.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T13:19:36.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.902+0000 7f0dcfe10640 1 -- 172.21.15.31:0/3091692231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc8107da0 msgr2=0x7f0dc810a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:36.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.902+0000 7f0dcfe10640 1 --2- 172.21.15.31:0/3091692231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc8107da0 0x7f0dc810a190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009ca0 tx=0x7f0dc002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.902+0000 7f0dcfe10640 1 -- 172.21.15.31:0/3091692231 shutdown_connections 2024-04-25T13:19:36.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.902+0000 7f0dcfe10640 1 --2- 172.21.15.31:0/3091692231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc8107da0 0x7f0dc810a190 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.902+0000 7f0dcfe10640 1 --2- 172.21.15.31:0/3091692231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dc8105470 0x7f0dc8107860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.902+0000 7f0dcfe10640 1 --2- 172.21.15.31:0/3091692231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc8102b40 0x7f0dc8104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.902+0000 7f0dcfe10640 1 -- 172.21.15.31:0/3091692231 >> 172.21.15.31:0/3091692231 conn(0x7f0dc80fc4d0 msgr2=0x7f0dc80fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.902+0000 7f0dcfe10640 1 -- 172.21.15.31:0/3091692231 shutdown_connections 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.902+0000 7f0dcfe10640 1 -- 172.21.15.31:0/3091692231 wait complete. 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcfe10640 1 Processor -- start 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcfe10640 1 -- start start 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcfe10640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dc8102b40 0x7f0dc8198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcfe10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc8105470 0x7f0dc8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcfe10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc8107da0 0x7f0dc819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcfe10640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc810db30 con 0x7f0dc8105470 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcfe10640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc810d9b0 con 0x7f0dc8107da0 2024-04-25T13:19:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcfe10640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc810dcb0 con 0x7f0dc8102b40 2024-04-25T13:19:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcd3ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc8105470 0x7f0dc8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcd3ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc8105470 0x7f0dc8198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40224/0 (socket says 172.21.15.31:40224) 2024-04-25T13:19:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcd3ab640 1 -- 172.21.15.31:0/2833754015 learned_addr learned my addr 172.21.15.31:0/2833754015 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcdbac640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dc8102b40 0x7f0dc8198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dce3ad640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc8107da0 0x7f0dc819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcd3ab640 1 -- 172.21.15.31:0/2833754015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dc8102b40 msgr2=0x7f0dc8198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcd3ab640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dc8102b40 0x7f0dc8198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcd3ab640 1 -- 172.21.15.31:0/2833754015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc8107da0 msgr2=0x7f0dc819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcd3ab640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc8107da0 0x7f0dc819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcd3ab640 1 -- 172.21.15.31:0/2833754015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc819fd30 con 0x7f0dc8105470 2024-04-25T13:19:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.903+0000 7f0dcdbac640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dc8102b40 0x7f0dc8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:36.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.904+0000 7f0dcd3ab640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc8105470 0x7f0dc8198650 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0db800d740 tx=0x7f0db800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:36.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.904+0000 7f0dbeffd640 1 -- 172.21.15.31:0/2833754015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8017070 con 0x7f0dc8105470 2024-04-25T13:19:36.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.904+0000 7f0dbeffd640 1 -- 172.21.15.31:0/2833754015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db800de50 con 0x7f0dc8105470 2024-04-25T13:19:36.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.904+0000 7f0dcfe10640 1 -- 172.21.15.31:0/2833754015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc819ffc0 con 0x7f0dc8105470 2024-04-25T13:19:36.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.904+0000 7f0dbeffd640 1 -- 172.21.15.31:0/2833754015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8004ce0 con 0x7f0dc8105470 2024-04-25T13:19:36.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.904+0000 7f0dcfe10640 1 -- 172.21.15.31:0/2833754015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc81a0490 con 0x7f0dc8105470 2024-04-25T13:19:36.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.905+0000 7f0dcfe10640 1 -- 172.21.15.31:0/2833754015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc80fdfc0 con 0x7f0dc8105470 2024-04-25T13:19:36.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.906+0000 7f0dbeffd640 1 -- 172.21.15.31:0/2833754015 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f0db8004e80 con 0x7f0dc8105470 2024-04-25T13:19:36.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.906+0000 7f0dbeffd640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0db40a3a60 0x7f0db40a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:36.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.907+0000 7f0dbeffd640 1 -- 172.21.15.31:0/2833754015 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8527+0+0 (secure 0 0 0) 0x7f0db80cfb70 con 0x7f0dc8105470 2024-04-25T13:19:36.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.907+0000 7f0dcdbac640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0db40a3a60 0x7f0db40a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:36.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.907+0000 7f0dcdbac640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0db40a3a60 0x7f0db40a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40058c0 tx=0x7f0dc4005850 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:36.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:36.911+0000 7f0dbeffd640 1 -- 172.21.15.31:0/2833754015 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0db809c720 con 0x7f0dc8105470 2024-04-25T13:19:37.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.140+0000 7f0dcfe10640 1 -- 172.21.15.31:0/2833754015 --> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f0dc8062390 con 0x7f0db40a3a60 2024-04-25T13:19:37.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.148+0000 7f0dbeffd640 1 -- 172.21.15.31:0/2833754015 <== mgr.24922 v2:172.21.15.31:6800/332946768 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f0dc8062390 con 0x7f0db40a3a60 2024-04-25T13:19:37.148 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T13:19:36", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T13:19:37.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 -- 172.21.15.31:0/2833754015 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0db40a3a60 msgr2=0x7f0db40a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:37.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0db40a3a60 0x7f0db40a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40058c0 tx=0x7f0dc4005850 comp rx=0 tx=0).stop 2024-04-25T13:19:37.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 -- 172.21.15.31:0/2833754015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc8105470 msgr2=0x7f0dc8198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:37.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc8105470 0x7f0dc8198650 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0db800d740 tx=0x7f0db800dc00 comp rx=0 tx=0).stop 2024-04-25T13:19:37.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 -- 172.21.15.31:0/2833754015 shutdown_connections 2024-04-25T13:19:37.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0db40a3a60 0x7f0db40a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0dc8107da0 0x7f0dc819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc8105470 0x7f0dc8198650 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 --2- 172.21.15.31:0/2833754015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dc8102b40 0x7f0dc8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 -- 172.21.15.31:0/2833754015 >> 172.21.15.31:0/2833754015 conn(0x7f0dc80fc4d0 msgr2=0x7f0dc8103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:37.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 -- 172.21.15.31:0/2833754015 shutdown_connections 2024-04-25T13:19:37.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.150+0000 7f0dcfe10640 1 -- 172.21.15.31:0/2833754015 wait complete. 2024-04-25T13:19:37.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T13:19:37.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.275+0000 7fceea48a640 1 -- 172.21.15.31:0/1036464410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcee4102a30 msgr2=0x7fcee4104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:37.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.275+0000 7fceea48a640 1 --2- 172.21.15.31:0/1036464410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcee4102a30 0x7fcee4104e20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009900 tx=0x7fcecc02a410 comp rx=0 tx=0).stop 2024-04-25T13:19:37.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.275+0000 7fceea48a640 1 -- 172.21.15.31:0/1036464410 shutdown_connections 2024-04-25T13:19:37.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.275+0000 7fceea48a640 1 --2- 172.21.15.31:0/1036464410 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcee4107c90 0x7fcee410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.275+0000 7fceea48a640 1 --2- 172.21.15.31:0/1036464410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee4105360 0x7fcee4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.275+0000 7fceea48a640 1 --2- 172.21.15.31:0/1036464410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcee4102a30 0x7fcee4104e20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.275+0000 7fceea48a640 1 -- 172.21.15.31:0/1036464410 >> 172.21.15.31:0/1036464410 conn(0x7fcee40fc420 msgr2=0x7fcee40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:37.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.275+0000 7fceea48a640 1 -- 172.21.15.31:0/1036464410 shutdown_connections 2024-04-25T13:19:37.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.275+0000 7fceea48a640 1 -- 172.21.15.31:0/1036464410 wait complete. 2024-04-25T13:19:37.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.275+0000 7fceea48a640 1 Processor -- start 2024-04-25T13:19:37.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fceea48a640 1 -- start start 2024-04-25T13:19:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fceea48a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcee4102a30 0x7fcee4198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fceea48a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcee4105360 0x7fcee4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fceea48a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee4107c90 0x7fcee419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fceea48a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcee410daf0 con 0x7fcee4105360 2024-04-25T13:19:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fceea48a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcee410d970 con 0x7fcee4102a30 2024-04-25T13:19:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fceea48a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcee410dc70 con 0x7fcee4107c90 2024-04-25T13:19:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fcee37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcee4105360 0x7fcee4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fcee37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcee4105360 0x7fcee4198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40240/0 (socket says 172.21.15.31:40240) 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fcee3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcee4102a30 0x7fcee4198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fcee37fe640 1 -- 172.21.15.31:0/3567586936 learned_addr learned my addr 172.21.15.31:0/3567586936 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fcee8a27640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee4107c90 0x7fcee419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fcee37fe640 1 -- 172.21.15.31:0/3567586936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee4107c90 msgr2=0x7fcee419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fcee37fe640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee4107c90 0x7fcee419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fcee37fe640 1 -- 172.21.15.31:0/3567586936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcee4102a30 msgr2=0x7fcee4198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fcee37fe640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcee4102a30 0x7fcee4198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fcee37fe640 1 -- 172.21.15.31:0/3567586936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee419fd70 con 0x7fcee4105360 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.276+0000 7fcee3fff640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcee4102a30 0x7fcee4198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.277+0000 7fcee37fe640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcee4105360 0x7fcee4198690 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fced4005fd0 tx=0x7fced400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.277+0000 7fcee17fa640 1 -- 172.21.15.31:0/3567586936 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4017070 con 0x7fcee4105360 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.277+0000 7fcee17fa640 1 -- 172.21.15.31:0/3567586936 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced4009c30 con 0x7fcee4105360 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.277+0000 7fcee17fa640 1 -- 172.21.15.31:0/3567586936 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4003d30 con 0x7fcee4105360 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.277+0000 7fceea48a640 1 -- 172.21.15.31:0/3567586936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee41a0060 con 0x7fcee4105360 2024-04-25T13:19:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.277+0000 7fceea48a640 1 -- 172.21.15.31:0/3567586936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee41a05b0 con 0x7fcee4105360 2024-04-25T13:19:37.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.278+0000 7fceea48a640 1 -- 172.21.15.31:0/3567586936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee40fdf10 con 0x7fcee4105360 2024-04-25T13:19:37.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.281+0000 7fcee17fa640 1 -- 172.21.15.31:0/3567586936 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fced4003ed0 con 0x7fcee4105360 2024-04-25T13:19:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.281+0000 7fcee17fa640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fced00a3a60 0x7fced00a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.281+0000 7fcee17fa640 1 -- 172.21.15.31:0/3567586936 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8527+0+0 (secure 0 0 0) 0x7fced40d04f0 con 0x7fcee4105360 2024-04-25T13:19:37.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.281+0000 7fcee3fff640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fced00a3a60 0x7fced00a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.282+0000 7fcee3fff640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fced00a3a60 0x7fced00a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcecc000c00 tx=0x7fcecc0314a0 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:37.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.288+0000 7fcee17fa640 1 -- 172.21.15.31:0/3567586936 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fced4020030 con 0x7fcee4105360 2024-04-25T13:19:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.306+0000 7fcee17fa640 1 -- 172.21.15.31:0/3567586936 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fced4099010 con 0x7fcee4105360 2024-04-25T13:19:37.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:37 smithi031 ceph-mon[29457]: [25/Apr/2024:13:19:36] ENGINE Bus STARTING 2024-04-25T13:19:37.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:37 smithi031 ceph-mon[29457]: mgrmap e22: x(active, since 1.0184s), standbys: y, z 2024-04-25T13:19:37.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:37 smithi031 ceph-mon[29457]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:37.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:37 smithi031 ceph-mon[29457]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:37.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:37 smithi031 ceph-mon[29457]: [25/Apr/2024:13:19:36] ENGINE Serving on http://172.21.15.31:8765 2024-04-25T13:19:37.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:37 smithi031 ceph-mon[29457]: [25/Apr/2024:13:19:36] ENGINE Serving on https://172.21.15.31:7150 2024-04-25T13:19:37.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:37 smithi031 ceph-mon[29457]: [25/Apr/2024:13:19:36] ENGINE Bus STARTED 2024-04-25T13:19:37.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:37 smithi031 ceph-mon[29457]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.536+0000 7fceea48a640 1 -- 172.21.15.31:0/3567586936 --> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fcee4062390 con 0x7fced00a3a60 2024-04-25T13:19:37.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.543+0000 7fcee17fa640 1 -- 172.21.15.31:0/3567586936 <== mgr.24922 v2:172.21.15.31:6800/332946768 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fcee4062390 con 0x7fced00a3a60 2024-04-25T13:19:37.546 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1m 6m3h 2024-04-25T13:19:37.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 -- 172.21.15.31:0/3567586936 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fced00a3a60 msgr2=0x7fced00a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:37.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fced00a3a60 0x7fced00a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcecc000c00 tx=0x7fcecc0314a0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 -- 172.21.15.31:0/3567586936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcee4105360 msgr2=0x7fcee4198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:37.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcee4105360 0x7fcee4198690 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fced4005fd0 tx=0x7fced400dde0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 -- 172.21.15.31:0/3567586936 shutdown_connections 2024-04-25T13:19:37.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fced00a3a60 0x7fced00a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee4107c90 0x7fcee419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcee4105360 0x7fcee4198690 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 --2- 172.21.15.31:0/3567586936 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcee4102a30 0x7fcee4198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:37.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 -- 172.21.15.31:0/3567586936 >> 172.21.15.31:0/3567586936 conn(0x7fcee40fc420 msgr2=0x7fcee41035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:37.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 -- 172.21.15.31:0/3567586936 shutdown_connections 2024-04-25T13:19:37.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:19:37.545+0000 7fceea48a640 1 -- 172.21.15.31:0/3567586936 wait complete. 2024-04-25T13:19:37.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T13:19:37.583 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T13:19:37.594 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T13:19:37.604 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T13:19:37.604 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T13:19:37.627 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T13:19:37.639 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/quiescer.py", line 387, in task cluster_name = config.get('cluster', 'ceph') AttributeError: 'NoneType' object has no attribute 'get' 2024-04-25T13:19:37.863 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=460927445d5d47519d04b22f4c6cf1fd Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/quiescer.py", line 387, in task cluster_name = config.get('cluster', 'ceph') AttributeError: 'NoneType' object has no attribute 'get' 2024-04-25T13:19:37.865 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T13:19:37.875 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T13:19:37.884 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T13:19:37.888 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:37 smithi136 ceph-mon[33038]: [25/Apr/2024:13:19:36] ENGINE Bus STARTING 2024-04-25T13:19:37.888 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:37 smithi136 ceph-mon[33038]: mgrmap e22: x(active, since 1.0184s), standbys: y, z 2024-04-25T13:19:37.888 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:37 smithi136 ceph-mon[33038]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:37.888 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:37 smithi136 ceph-mon[33038]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:37.888 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:37 smithi136 ceph-mon[33038]: [25/Apr/2024:13:19:36] ENGINE Serving on http://172.21.15.31:8765 2024-04-25T13:19:37.888 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:37 smithi136 ceph-mon[33038]: [25/Apr/2024:13:19:36] ENGINE Serving on https://172.21.15.31:7150 2024-04-25T13:19:37.888 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:37 smithi136 ceph-mon[33038]: [25/Apr/2024:13:19:36] ENGINE Bus STARTED 2024-04-25T13:19:37.889 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:37 smithi136 ceph-mon[33038]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:37.889 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:37 smithi066 ceph-mon[33137]: [25/Apr/2024:13:19:36] ENGINE Bus STARTING 2024-04-25T13:19:37.889 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:37 smithi066 ceph-mon[33137]: mgrmap e22: x(active, since 1.0184s), standbys: y, z 2024-04-25T13:19:37.889 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:37 smithi066 ceph-mon[33137]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:37.889 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:37 smithi066 ceph-mon[33137]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:37.889 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:37 smithi066 ceph-mon[33137]: [25/Apr/2024:13:19:36] ENGINE Serving on http://172.21.15.31:8765 2024-04-25T13:19:37.889 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:37 smithi066 ceph-mon[33137]: [25/Apr/2024:13:19:36] ENGINE Serving on https://172.21.15.31:7150 2024-04-25T13:19:37.889 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:37 smithi066 ceph-mon[33137]: [25/Apr/2024:13:19:36] ENGINE Bus STARTED 2024-04-25T13:19:37.889 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:37 smithi066 ceph-mon[33137]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:37.898 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T13:19:37.908 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T13:19:37.908 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:19:37.909 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T13:19:37.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T13:19:37.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T13:19:38.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:38 smithi066 ceph-mon[33137]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:38.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:38 smithi066 ceph-mon[33137]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:38.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:38 smithi066 ceph-mon[33137]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T13:19:38.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:38 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:38 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:38 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:38 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:38 smithi066 ceph-mon[33137]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:38.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:38 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:38 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:38 smithi136 ceph-mon[33038]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:38.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:38 smithi136 ceph-mon[33038]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:38.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:38 smithi136 ceph-mon[33038]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T13:19:38.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:38 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:38 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:38 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:38 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:38 smithi136 ceph-mon[33038]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:38.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:38 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.720 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:38 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:38 smithi031 ceph-mon[29457]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:38.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:38 smithi031 ceph-mon[29457]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:38.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:38 smithi031 ceph-mon[29457]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T13:19:38.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:38 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:38 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:38 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:38 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:38 smithi031 ceph-mon[29457]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:38.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:38 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:38.735 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:38 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T13:19:39.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Unable to set osd_memory_target on smithi136 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:19:39.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-25T13:19:39.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T13:19:39.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.911 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.911 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.911 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi031 to 177.2M 2024-04-25T13:19:39.911 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Unable to set osd_memory_target on smithi031 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:19:39.911 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:39.911 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:39.911 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T13:19:39.911 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Updating smithi066:/etc/ceph/ceph.conf 2024-04-25T13:19:39.911 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T13:19:39.911 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Updating smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:19:39.912 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Updating smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:19:39.912 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:39 smithi066 ceph-mon[33137]: Updating smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:19:39.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T13:19:39.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Unable to set osd_memory_target on smithi136 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:19:39.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-25T13:19:39.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T13:19:39.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi031 to 177.2M 2024-04-25T13:19:39.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Unable to set osd_memory_target on smithi031 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:19:39.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:39.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:39.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T13:19:39.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Updating smithi066:/etc/ceph/ceph.conf 2024-04-25T13:19:39.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T13:19:39.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Updating smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:19:39.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Updating smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:19:39.972 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:39 smithi136 ceph-mon[33038]: Updating smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:19:39.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T13:19:39.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Unable to set osd_memory_target on smithi136 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:19:39.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-25T13:19:39.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T13:19:39.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:39.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:39.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi031 to 177.2M 2024-04-25T13:19:39.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Unable to set osd_memory_target on smithi031 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:19:39.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:39.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:39.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-25T13:19:39.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Updating smithi066:/etc/ceph/ceph.conf 2024-04-25T13:19:39.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T13:19:39.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Updating smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:19:39.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Updating smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:19:39.987 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:39 smithi031 ceph-mon[29457]: Updating smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:19:40.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:19:40.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:19:40.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:19:40.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:40.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: Updating smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:19:40.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: Updating smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:19:40.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: Updating smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:19:40.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T13:19:40.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:40.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:40.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:40.910 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:19:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:19:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:19:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: Updating smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:19:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: Updating smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:19:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: Updating smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:19:40.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T13:19:40.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:40.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:40.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:40.971 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:19:40.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:19:40.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:19:40.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:40.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: Updating smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:19:40.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: Updating smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:19:40.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: Updating smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:19:40.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T13:19:40.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:40.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:40.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:40.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:40.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:19:42.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:42 smithi066 ceph-mon[33137]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:42.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:42 smithi136 ceph-mon[33038]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:42.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:42 smithi031 ceph-mon[29457]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:19:44.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:44 smithi066 ceph-mon[33137]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-04-25T13:19:44.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:44 smithi136 ceph-mon[33038]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-04-25T13:19:44.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:44 smithi031 ceph-mon[29457]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-04-25T13:19:46.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:46 smithi066 ceph-mon[33137]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-04-25T13:19:46.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:46 smithi136 ceph-mon[33038]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-04-25T13:19:46.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:46 smithi031 ceph-mon[29457]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-04-25T13:19:48.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:48 smithi066 ceph-mon[33137]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-04-25T13:19:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:48 smithi136 ceph-mon[33038]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-04-25T13:19:48.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:48 smithi031 ceph-mon[29457]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-04-25T13:19:50.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:50 smithi066 ceph-mon[33137]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:19:50.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:50 smithi136 ceph-mon[33038]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:19:50.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:50 smithi031 ceph-mon[29457]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:19:52.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:52 smithi066 ceph-mon[33137]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:19:52.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:52 smithi136 ceph-mon[33038]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:19:52.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:52 smithi031 ceph-mon[29457]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:19:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:54 smithi066 ceph-mon[33137]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-04-25T13:19:54.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:54 smithi136 ceph-mon[33038]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-04-25T13:19:54.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:54 smithi031 ceph-mon[29457]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-04-25T13:19:56.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:56 smithi066 ceph-mon[33137]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:56.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:56 smithi136 ceph-mon[33038]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:56.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:56 smithi031 ceph-mon[29457]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:19:58.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:19:58 smithi066 ceph-mon[33137]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:19:58.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:19:58 smithi136 ceph-mon[33038]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:19:58.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:19:58 smithi031 ceph-mon[29457]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:00.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:00 smithi066 ceph-mon[33137]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:00.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:00 smithi066 ceph-mon[33137]: overall HEALTH_OK 2024-04-25T13:20:00.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:00 smithi136 ceph-mon[33038]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:00.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:00 smithi136 ceph-mon[33038]: overall HEALTH_OK 2024-04-25T13:20:00.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:00 smithi031 ceph-mon[29457]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:00.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:00 smithi031 ceph-mon[29457]: overall HEALTH_OK 2024-04-25T13:20:00.984 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:20:00 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:20:00.795+0000 7f574e5df700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:20:00.985 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:20:00 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:20:00.795+0000 7f574e5df700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:20:00.985 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:20:00 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:20:00.795+0000 7f574e5df700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:20:00.985 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:20:00 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:20:00.795+0000 7f574e5df700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:20:00.985 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:20:00 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x[29687]: 2024-04-25T13:20:00.795+0000 7f574e5df700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:20:02.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:02 smithi066 ceph-mon[33137]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 11 KiB/s wr, 3 op/s 2024-04-25T13:20:02.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:02 smithi066 ceph-mon[33137]: mgrmap e25: x(active, since 26s), standbys: y, z 2024-04-25T13:20:02.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:02 smithi136 ceph-mon[33038]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 11 KiB/s wr, 3 op/s 2024-04-25T13:20:02.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:02 smithi136 ceph-mon[33038]: mgrmap e25: x(active, since 26s), standbys: y, z 2024-04-25T13:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:02 smithi031 ceph-mon[29457]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 11 KiB/s wr, 3 op/s 2024-04-25T13:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:02 smithi031 ceph-mon[29457]: mgrmap e25: x(active, since 26s), standbys: y, z 2024-04-25T13:20:04.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:04 smithi066 ceph-mon[33137]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:20:04.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:04 smithi136 ceph-mon[33038]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:20:04.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:04 smithi031 ceph-mon[29457]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:20:06.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:06 smithi136 ceph-mon[33038]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:20:06.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:06 smithi031 ceph-mon[29457]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:20:07.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:06 smithi066 ceph-mon[33137]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:20:08.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:08 smithi136 ceph-mon[33038]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:20:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:08 smithi031 ceph-mon[29457]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:20:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:08 smithi066 ceph-mon[33137]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:20:10.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:10 smithi136 ceph-mon[33038]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:20:10.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:10 smithi031 ceph-mon[29457]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:20:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:10 smithi066 ceph-mon[33137]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:20:12.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:12 smithi136 ceph-mon[33038]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:20:12.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:12 smithi031 ceph-mon[29457]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:20:13.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:12 smithi066 ceph-mon[33137]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:20:14.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:14 smithi136 ceph-mon[33038]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T13:20:14.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:14 smithi031 ceph-mon[29457]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T13:20:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:14 smithi066 ceph-mon[33137]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T13:20:16.960 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:16 smithi066 ceph-mon[33137]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:20:16.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:16 smithi136 ceph-mon[33038]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:20:16.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:16 smithi031 ceph-mon[29457]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:20:18.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:18 smithi136 ceph-mon[33038]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:20:18.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:18 smithi031 ceph-mon[29457]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:20:19.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:18 smithi066 ceph-mon[33137]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:20:20.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:20 smithi136 ceph-mon[33038]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:20:20.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:20 smithi031 ceph-mon[29457]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:20:21.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:20 smithi066 ceph-mon[33137]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:20:22.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:22 smithi136 ceph-mon[33038]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:20:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:22 smithi031 ceph-mon[29457]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:20:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:22 smithi066 ceph-mon[33137]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:20:24.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:24 smithi136 ceph-mon[33038]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:20:24.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:24 smithi031 ceph-mon[29457]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:20:25.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:24 smithi066 ceph-mon[33137]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:20:26.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:26 smithi136 ceph-mon[33038]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:26.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:26 smithi031 ceph-mon[29457]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:27.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:26 smithi066 ceph-mon[33137]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:28.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:28 smithi136 ceph-mon[33038]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:28.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:28 smithi031 ceph-mon[29457]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:29.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:28 smithi066 ceph-mon[33137]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:30.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:30 smithi136 ceph-mon[33038]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:30.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:30 smithi031 ceph-mon[29457]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:31.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:30 smithi066 ceph-mon[33137]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:32.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:32 smithi136 ceph-mon[33038]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:32.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:32 smithi031 ceph-mon[29457]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:33.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:32 smithi066 ceph-mon[33137]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:34.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:34 smithi136 ceph-mon[33038]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:34.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:34 smithi031 ceph-mon[29457]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:35.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:34 smithi066 ceph-mon[33137]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T13:20:35.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:35 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T13:20:35.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T13:20:35.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:35 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T13:20:36.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T13:20:36.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:35 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T13:20:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:36 smithi031 ceph-mon[29457]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:36 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T13:20:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:36 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T13:20:36.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:36 smithi031 ceph-mon[29457]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:20:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:36 smithi066 ceph-mon[33137]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:36 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T13:20:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:36 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T13:20:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:36 smithi066 ceph-mon[33137]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:20:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:36 smithi136 ceph-mon[33038]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:36 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T13:20:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:36 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T13:20:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:36 smithi136 ceph-mon[33038]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:20:37.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:37 smithi031 ceph-mon[29457]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:20:38.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:37 smithi066 ceph-mon[33137]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:20:38.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:37 smithi136 ceph-mon[33038]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:20:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:38 smithi066 ceph-mon[33137]: pgmap v36: 97 pgs: 2 peering, 95 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:38 smithi066 ceph-mon[33137]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:20:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:38 smithi136 ceph-mon[33038]: pgmap v36: 97 pgs: 2 peering, 95 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:38 smithi136 ceph-mon[33038]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:20:39.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:38 smithi031 ceph-mon[29457]: pgmap v36: 97 pgs: 2 peering, 95 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:39.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:38 smithi031 ceph-mon[29457]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:20:40.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:39 smithi066 ceph-mon[33137]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:20:40.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:39 smithi136 ceph-mon[33038]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:20:40.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:39 smithi031 ceph-mon[29457]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:20:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:40 smithi066 ceph-mon[33137]: pgmap v39: 97 pgs: 2 peering, 95 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:40 smithi066 ceph-mon[33137]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:20:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:40 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:40 smithi136 ceph-mon[33038]: pgmap v39: 97 pgs: 2 peering, 95 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:40 smithi136 ceph-mon[33038]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:20:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:40 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:41.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:40 smithi031 ceph-mon[29457]: pgmap v39: 97 pgs: 2 peering, 95 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:41.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:40 smithi031 ceph-mon[29457]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:20:41.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:40 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:41 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:20:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:41 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:20:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:41 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:41 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:42.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:41 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:20:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:41 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:20:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:41 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:20:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:41 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:41 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:41 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:20:42.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:41 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:20:42.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:41 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:20:42.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:41 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:42.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:41 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:42.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:41 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:20:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:42 smithi066 ceph-mon[33137]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 84 KiB/s, 3 keys/s, 0 objects/s recovering 2024-04-25T13:20:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:42 smithi136 ceph-mon[33038]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 84 KiB/s, 3 keys/s, 0 objects/s recovering 2024-04-25T13:20:43.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:42 smithi031 ceph-mon[29457]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 84 KiB/s, 3 keys/s, 0 objects/s recovering 2024-04-25T13:20:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:44 smithi066 ceph-mon[33137]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 64 KiB/s, 2 keys/s, 0 objects/s recovering 2024-04-25T13:20:45.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:44 smithi136 ceph-mon[33038]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 64 KiB/s, 2 keys/s, 0 objects/s recovering 2024-04-25T13:20:45.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:44 smithi031 ceph-mon[29457]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 64 KiB/s, 2 keys/s, 0 objects/s recovering 2024-04-25T13:20:47.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:46 smithi066 ceph-mon[33137]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 675 B/s rd, 1 op/s; 51 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:46 smithi136 ceph-mon[33038]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 675 B/s rd, 1 op/s; 51 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:47.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:46 smithi031 ceph-mon[29457]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 675 B/s rd, 1 op/s; 51 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:48 smithi066 ceph-mon[33137]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 45 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:49.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:48 smithi136 ceph-mon[33038]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 45 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:49.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:48 smithi031 ceph-mon[29457]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s; 45 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:51.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:50 smithi066 ceph-mon[33137]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s; 38 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:51.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:50 smithi136 ceph-mon[33038]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s; 38 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:51.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:50 smithi031 ceph-mon[29457]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s; 38 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:52 smithi066 ceph-mon[33137]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 33 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:53.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:52 smithi136 ceph-mon[33038]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 33 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:53.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:52 smithi031 ceph-mon[29457]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 33 KiB/s, 1 keys/s, 0 objects/s recovering 2024-04-25T13:20:55.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:54 smithi066 ceph-mon[33137]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:55.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:54 smithi136 ceph-mon[33038]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:55.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:54 smithi031 ceph-mon[29457]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:57.036 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:56 smithi031 ceph-mon[29457]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:56 smithi066 ceph-mon[33137]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:56 smithi136 ceph-mon[33038]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:20:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:20:58 smithi066 ceph-mon[33137]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:20:58 smithi136 ceph-mon[33038]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:20:59.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:20:58 smithi031 ceph-mon[29457]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:01.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:00 smithi066 ceph-mon[33137]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:01.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:00 smithi136 ceph-mon[33038]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:01.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:00 smithi031 ceph-mon[29457]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:02 smithi066 ceph-mon[33137]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:02 smithi136 ceph-mon[33038]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:03.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:02 smithi031 ceph-mon[29457]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:05.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:04 smithi066 ceph-mon[33137]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:21:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:04 smithi136 ceph-mon[33038]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:21:05.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:04 smithi031 ceph-mon[29457]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:21:07.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:06 smithi066 ceph-mon[33137]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:21:07.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:06 smithi136 ceph-mon[33038]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:21:07.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:06 smithi031 ceph-mon[29457]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:21:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:08 smithi066 ceph-mon[33137]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:08 smithi066 ceph-mon[33137]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T13:21:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:08 smithi136 ceph-mon[33038]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:08 smithi136 ceph-mon[33038]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T13:21:09.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:08 smithi031 ceph-mon[29457]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:09.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:08 smithi031 ceph-mon[29457]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T13:21:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:10 smithi066 ceph-mon[33137]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:10 smithi136 ceph-mon[33038]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:11.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:10 smithi031 ceph-mon[29457]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:13.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:12 smithi066 ceph-mon[33137]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:13.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:12 smithi136 ceph-mon[33038]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:13.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:12 smithi031 ceph-mon[29457]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:21:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:14 smithi066 ceph-mon[33137]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:21:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:14 smithi136 ceph-mon[33038]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:21:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:14 smithi031 ceph-mon[29457]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:21:17.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:16 smithi066 ceph-mon[33137]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:21:17.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:16 smithi136 ceph-mon[33038]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:21:17.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:16 smithi031 ceph-mon[29457]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:21:19.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:18 smithi066 ceph-mon[33137]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-25T13:21:19.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:18 smithi136 ceph-mon[33038]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-25T13:21:19.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:18 smithi031 ceph-mon[29457]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-25T13:21:21.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:20 smithi066 ceph-mon[33137]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:21.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:20 smithi136 ceph-mon[33038]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:21.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:20 smithi031 ceph-mon[29457]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:23.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:22 smithi066 ceph-mon[33137]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:22 smithi136 ceph-mon[33038]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:23.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:22 smithi031 ceph-mon[29457]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:25.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:24 smithi066 ceph-mon[33137]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:24 smithi136 ceph-mon[33038]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:24 smithi031 ceph-mon[29457]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:27.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:26 smithi066 ceph-mon[33137]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:27.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:26 smithi136 ceph-mon[33038]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:27.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:26 smithi031 ceph-mon[29457]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:29.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:28 smithi066 ceph-mon[33137]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:29.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:28 smithi136 ceph-mon[33038]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:29.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:28 smithi031 ceph-mon[29457]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:31.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:30 smithi066 ceph-mon[33137]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:31.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:30 smithi136 ceph-mon[33038]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:31.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:30 smithi031 ceph-mon[29457]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:33.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:32 smithi066 ceph-mon[33137]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:32 smithi136 ceph-mon[33038]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:33.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:32 smithi031 ceph-mon[29457]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:35.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:34 smithi066 ceph-mon[33137]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:35.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:34 smithi136 ceph-mon[33038]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:35.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:34 smithi031 ceph-mon[29457]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:36 smithi066 ceph-mon[33137]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:36 smithi136 ceph-mon[33038]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:37.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:36 smithi031 ceph-mon[29457]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:38 smithi066 ceph-mon[33137]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:38 smithi136 ceph-mon[33038]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:39.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:38 smithi031 ceph-mon[29457]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:40 smithi066 ceph-mon[33137]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:40 smithi136 ceph-mon[33038]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:41.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:40 smithi031 ceph-mon[29457]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:42.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:41 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:42.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:41 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:42.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:41 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:42 smithi066 ceph-mon[33137]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:42 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:42 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:42 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:42 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:42 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:43.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:42 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:43.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:42 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:42 smithi136 ceph-mon[33038]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:42 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:42 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:42 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:42 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:42 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:42 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:42 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:42 smithi031 ceph-mon[29457]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:42 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:42 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:42 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:42 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:42 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:42 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:42 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:21:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:44 smithi066 ceph-mon[33137]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:45.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:44 smithi136 ceph-mon[33038]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:45.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:44 smithi031 ceph-mon[29457]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:47.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:46 smithi066 ceph-mon[33137]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:46 smithi136 ceph-mon[33038]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:47.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:46 smithi031 ceph-mon[29457]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:48 smithi066 ceph-mon[33137]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:49.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:48 smithi136 ceph-mon[33038]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:49.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:48 smithi031 ceph-mon[29457]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:51.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:50 smithi066 ceph-mon[33137]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:51.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:50 smithi136 ceph-mon[33038]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:51.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:50 smithi031 ceph-mon[29457]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:52 smithi066 ceph-mon[33137]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:53.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:52 smithi136 ceph-mon[33038]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:53.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:52 smithi031 ceph-mon[29457]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:55.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:54 smithi066 ceph-mon[33137]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:55.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:54 smithi136 ceph-mon[33038]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:55.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:54 smithi031 ceph-mon[29457]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:56 smithi066 ceph-mon[33137]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:56 smithi136 ceph-mon[33038]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:57.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:56 smithi031 ceph-mon[29457]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:21:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:21:58 smithi066 ceph-mon[33137]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:21:58 smithi136 ceph-mon[33038]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:21:59.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:21:58 smithi031 ceph-mon[29457]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:01.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:00 smithi066 ceph-mon[33137]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:01.218 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:00 smithi136 ceph-mon[33038]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:01.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:00 smithi031 ceph-mon[29457]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:02 smithi066 ceph-mon[33137]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:22:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:02 smithi136 ceph-mon[33038]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:22:03.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:02 smithi031 ceph-mon[29457]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:22:05.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:04 smithi066 ceph-mon[33137]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:22:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:04 smithi136 ceph-mon[33038]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:22:05.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:04 smithi031 ceph-mon[29457]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:22:07.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:06 smithi066 ceph-mon[33137]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:22:07.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:06 smithi136 ceph-mon[33038]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:22:07.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:06 smithi031 ceph-mon[29457]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:22:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:08 smithi066 ceph-mon[33137]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:22:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:08 smithi136 ceph-mon[33038]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:22:09.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:08 smithi031 ceph-mon[29457]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:22:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:10 smithi136 ceph-mon[33038]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:22:11.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:10 smithi031 ceph-mon[29457]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:22:11.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:10 smithi066 ceph-mon[33137]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:22:13.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:12 smithi136 ceph-mon[33038]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:22:13.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:12 smithi031 ceph-mon[29457]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:22:13.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:12 smithi066 ceph-mon[33137]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:22:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:14 smithi136 ceph-mon[33038]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:22:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:14 smithi031 ceph-mon[29457]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:22:15.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:14 smithi066 ceph-mon[33137]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:22:17.210 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:16 smithi066 ceph-mon[33137]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:22:17.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:16 smithi136 ceph-mon[33038]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:22:17.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:16 smithi031 ceph-mon[29457]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:22:19.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:18 smithi136 ceph-mon[33038]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:22:19.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:18 smithi031 ceph-mon[29457]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:22:19.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:18 smithi066 ceph-mon[33137]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:22:21.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:20 smithi136 ceph-mon[33038]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:22:21.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:20 smithi031 ceph-mon[29457]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:22:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:20 smithi066 ceph-mon[33137]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:22:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:22 smithi136 ceph-mon[33038]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:22:23.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:22 smithi031 ceph-mon[29457]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:22:23.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:22 smithi066 ceph-mon[33137]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:22:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:24 smithi136 ceph-mon[33038]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:25.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:24 smithi031 ceph-mon[29457]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:24 smithi066 ceph-mon[33137]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:27.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:26 smithi136 ceph-mon[33038]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:26 smithi031 ceph-mon[29457]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:26 smithi066 ceph-mon[33137]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:29.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:28 smithi136 ceph-mon[33038]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:29.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:28 smithi031 ceph-mon[29457]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:29.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:28 smithi066 ceph-mon[33137]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:31.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:30 smithi136 ceph-mon[33038]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:31.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:30 smithi031 ceph-mon[29457]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:30 smithi066 ceph-mon[33137]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:32 smithi136 ceph-mon[33038]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:33.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:32 smithi031 ceph-mon[29457]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:32 smithi066 ceph-mon[33137]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:35.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:34 smithi136 ceph-mon[33038]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:35.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:34 smithi031 ceph-mon[29457]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:34 smithi066 ceph-mon[33137]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:36 smithi136 ceph-mon[33038]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:37.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:36 smithi031 ceph-mon[29457]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:37.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:36 smithi066 ceph-mon[33137]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:38 smithi136 ceph-mon[33038]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:39.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:38 smithi031 ceph-mon[29457]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:39.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:38 smithi066 ceph-mon[33137]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:40 smithi136 ceph-mon[33038]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:41.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:40 smithi031 ceph-mon[29457]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:41.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:40 smithi066 ceph-mon[33137]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:42 smithi136 ceph-mon[33038]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:43.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:42 smithi031 ceph-mon[29457]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:43.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:42 smithi066 ceph-mon[33137]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:44.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:43 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:44.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:43 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:44.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:43 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:44.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:43 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:22:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:43 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:43 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:43 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:44.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:43 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:22:44.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:43 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:44.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:43 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:44.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:43 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:44.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:43 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:22:45.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:44 smithi136 ceph-mon[33038]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:45.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:44 smithi031 ceph-mon[29457]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:45.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:44 smithi066 ceph-mon[33137]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:46 smithi136 ceph-mon[33038]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:47.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:46 smithi031 ceph-mon[29457]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:47.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:46 smithi066 ceph-mon[33137]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:49.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:48 smithi031 ceph-mon[29457]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:49.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:48 smithi066 ceph-mon[33137]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:48 smithi136 ceph-mon[33038]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:51.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:50 smithi031 ceph-mon[29457]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:50 smithi066 ceph-mon[33137]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:51.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:50 smithi136 ceph-mon[33038]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:53.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:52 smithi031 ceph-mon[29457]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:52 smithi066 ceph-mon[33137]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:53.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:52 smithi136 ceph-mon[33038]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:55.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:54 smithi031 ceph-mon[29457]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:55.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:54 smithi066 ceph-mon[33137]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:55.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:54 smithi136 ceph-mon[33038]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:57.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:56 smithi031 ceph-mon[29457]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:57.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:56 smithi066 ceph-mon[33137]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:57.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:56 smithi136 ceph-mon[33038]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:22:59.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:22:58 smithi066 ceph-mon[33137]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:59.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:22:58 smithi136 ceph-mon[33038]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:22:59.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:22:58 smithi031 ceph-mon[29457]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:01.270 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:00 smithi136 ceph-mon[33038]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-25T13:23:01.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:00 smithi066 ceph-mon[33137]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-25T13:23:01.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:00 smithi031 ceph-mon[29457]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-25T13:23:03.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:02 smithi066 ceph-mon[33137]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:23:03.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:02 smithi136 ceph-mon[33038]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:23:03.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:02 smithi031 ceph-mon[29457]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:23:05.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:04 smithi066 ceph-mon[33137]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:23:05.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:04 smithi136 ceph-mon[33038]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:23:05.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:04 smithi031 ceph-mon[29457]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:23:07.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:07 smithi066 ceph-mon[33137]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:23:07.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:07 smithi136 ceph-mon[33038]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:23:07.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:06 smithi031 ceph-mon[29457]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:23:09.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:09 smithi066 ceph-mon[33137]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:23:09.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:09 smithi136 ceph-mon[33038]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:23:09.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:09 smithi031 ceph-mon[29457]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:23:11.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:11 smithi066 ceph-mon[33137]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:23:11.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:11 smithi136 ceph-mon[33038]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:23:11.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:11 smithi031 ceph-mon[29457]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:23:13.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:13 smithi066 ceph-mon[33137]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:23:13.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:13 smithi136 ceph-mon[33038]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:23:13.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:13 smithi031 ceph-mon[29457]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:23:15.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:15 smithi066 ceph-mon[33137]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:23:15.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:15 smithi136 ceph-mon[33038]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:23:15.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:15 smithi031 ceph-mon[29457]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:23:17.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:17 smithi066 ceph-mon[33137]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:23:17.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:17 smithi136 ceph-mon[33038]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:23:17.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:17 smithi031 ceph-mon[29457]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:23:19.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:19 smithi066 ceph-mon[33137]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:23:19.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:19 smithi136 ceph-mon[33038]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:23:19.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:19 smithi031 ceph-mon[29457]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:23:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:21 smithi066 ceph-mon[33137]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:23:21.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:21 smithi136 ceph-mon[33038]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:23:21.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:21 smithi031 ceph-mon[29457]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:23:23.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:23 smithi066 ceph-mon[33137]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:23:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:23 smithi136 ceph-mon[33038]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:23:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:23 smithi031 ceph-mon[29457]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:23:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:25 smithi066 ceph-mon[33137]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:25.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:25 smithi136 ceph-mon[33038]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:25.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:25 smithi031 ceph-mon[29457]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:27.285 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:27 smithi031 ceph-mon[29457]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:27 smithi066 ceph-mon[33137]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:27 smithi136 ceph-mon[33038]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:29.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:29 smithi066 ceph-mon[33137]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:29.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:29 smithi136 ceph-mon[33038]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:29.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:29 smithi031 ceph-mon[29457]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:31 smithi066 ceph-mon[33137]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:31 smithi136 ceph-mon[33038]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:31 smithi031 ceph-mon[29457]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:33 smithi066 ceph-mon[33137]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:33 smithi136 ceph-mon[33038]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:33.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:33 smithi031 ceph-mon[29457]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:35 smithi066 ceph-mon[33137]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:35.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:35 smithi136 ceph-mon[33038]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:35.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:35 smithi031 ceph-mon[29457]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:37.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:37 smithi066 ceph-mon[33137]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:37.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:37 smithi136 ceph-mon[33038]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:37.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:37 smithi031 ceph-mon[29457]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:39.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:39 smithi066 ceph-mon[33137]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:39.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:39 smithi136 ceph-mon[33038]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:39.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:39 smithi031 ceph-mon[29457]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:41.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:41 smithi066 ceph-mon[33137]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:41.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:41 smithi136 ceph-mon[33038]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:41.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:41 smithi031 ceph-mon[29457]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:43.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:43 smithi066 ceph-mon[33137]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:43.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:43 smithi136 ceph-mon[33038]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:43.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:43 smithi031 ceph-mon[29457]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:44.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:44 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:44.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:44 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:44.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:44 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:45.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:45 smithi066 ceph-mon[33137]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:45.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:45 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:45.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:45 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:45.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:45 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:23:45.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:45 smithi136 ceph-mon[33038]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:45.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:45 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:45.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:45 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:45.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:45 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:23:45.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:45 smithi031 ceph-mon[29457]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:45.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:45 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:45.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:45 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:45.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:45 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:23:47.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:47 smithi066 ceph-mon[33137]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:47.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:47 smithi136 ceph-mon[33038]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:47.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:47 smithi031 ceph-mon[29457]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:49.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:49 smithi066 ceph-mon[33137]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:49 smithi136 ceph-mon[33038]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:49.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:49 smithi031 ceph-mon[29457]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:51 smithi066 ceph-mon[33137]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:51.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:51 smithi136 ceph-mon[33038]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:51.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:51 smithi031 ceph-mon[29457]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:52.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:52 smithi066 ceph-mon[33137]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:52.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:52 smithi136 ceph-mon[33038]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:52.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:52 smithi031 ceph-mon[29457]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:54.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:54 smithi136 ceph-mon[33038]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:54.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:54 smithi031 ceph-mon[29457]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:54 smithi066 ceph-mon[33137]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:56.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:56 smithi136 ceph-mon[33038]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:56.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:56 smithi031 ceph-mon[29457]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:56.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:56 smithi066 ceph-mon[33137]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:23:58.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:23:58 smithi136 ceph-mon[33038]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:58.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:23:58 smithi031 ceph-mon[29457]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:23:58.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:23:58 smithi066 ceph-mon[33137]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:00.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:00 smithi136 ceph-mon[33038]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:00.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:00 smithi031 ceph-mon[29457]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:00.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:00 smithi066 ceph-mon[33137]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:02.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:02 smithi136 ceph-mon[33038]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:24:02.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:02 smithi031 ceph-mon[29457]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:24:02.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:02 smithi066 ceph-mon[33137]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:24:04.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:04 smithi136 ceph-mon[33038]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:24:04.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:04 smithi031 ceph-mon[29457]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:24:04.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:04 smithi066 ceph-mon[33137]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:24:06.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:06 smithi136 ceph-mon[33038]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:24:06.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:06 smithi031 ceph-mon[29457]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:24:06.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:06 smithi066 ceph-mon[33137]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:24:08.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:08 smithi136 ceph-mon[33038]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:24:08.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:08 smithi031 ceph-mon[29457]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:24:08.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:08 smithi066 ceph-mon[33137]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:24:10.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:10 smithi136 ceph-mon[33038]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:24:10.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:10 smithi031 ceph-mon[29457]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:24:10.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:10 smithi066 ceph-mon[33137]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:24:12.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:12 smithi136 ceph-mon[33038]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:24:12.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:12 smithi031 ceph-mon[29457]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:24:12.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:12 smithi066 ceph-mon[33137]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:24:14.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:14 smithi136 ceph-mon[33038]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:24:14.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:14 smithi031 ceph-mon[29457]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:24:14.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:14 smithi066 ceph-mon[33137]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:24:16.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:16 smithi136 ceph-mon[33038]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:24:16.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:16 smithi031 ceph-mon[29457]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:24:16.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:16 smithi066 ceph-mon[33137]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:24:18.710 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:18 smithi066 ceph-mon[33137]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:24:18.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:18 smithi136 ceph-mon[33038]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:24:18.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:18 smithi031 ceph-mon[29457]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:24:20.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:20 smithi136 ceph-mon[33038]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:24:20.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:20 smithi031 ceph-mon[29457]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:24:20.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:20 smithi066 ceph-mon[33137]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:24:22.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:22 smithi136 ceph-mon[33038]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:24:22.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:22 smithi031 ceph-mon[29457]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:24:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:22 smithi066 ceph-mon[33137]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:24:24.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:24 smithi136 ceph-mon[33038]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:24.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:24 smithi031 ceph-mon[29457]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:24.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:24 smithi066 ceph-mon[33137]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:26.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:26 smithi136 ceph-mon[33038]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:26.733 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:26 smithi031 ceph-mon[29457]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:26.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:26 smithi066 ceph-mon[33137]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:28.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:28 smithi031 ceph-mon[29457]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:28.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:28 smithi066 ceph-mon[33137]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:28.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:28 smithi136 ceph-mon[33038]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:30.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:30 smithi031 ceph-mon[29457]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:30.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:30 smithi066 ceph-mon[33137]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:30.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:30 smithi136 ceph-mon[33038]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:32.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:32 smithi031 ceph-mon[29457]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:32.771 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:32 smithi136 ceph-mon[33038]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:32.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:32 smithi066 ceph-mon[33137]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:34.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:34 smithi031 ceph-mon[29457]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:34.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:34 smithi066 ceph-mon[33137]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:34.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:34 smithi136 ceph-mon[33038]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:36.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:36 smithi031 ceph-mon[29457]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:36.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:36 smithi066 ceph-mon[33137]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:36.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:36 smithi136 ceph-mon[33038]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:38.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:38 smithi066 ceph-mon[33137]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:38.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:38 smithi136 ceph-mon[33038]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:38.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:38 smithi031 ceph-mon[29457]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:40.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:40 smithi066 ceph-mon[33137]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:40.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:40 smithi136 ceph-mon[33038]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:40.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:40 smithi031 ceph-mon[29457]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:42.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:42 smithi066 ceph-mon[33137]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:42.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:42 smithi136 ceph-mon[33038]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:42.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:42 smithi031 ceph-mon[29457]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:44.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:44 smithi066 ceph-mon[33137]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:44.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:44 smithi136 ceph-mon[33038]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:44.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:44 smithi031 ceph-mon[29457]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:45.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:45 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:24:45.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:45 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:45.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:45 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:24:45.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:45 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:24:45.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:45 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:24:45.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:45 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:45.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:45 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:24:45.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:45 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:24:45.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:45 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:24:45.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:45 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:45.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:45 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:24:45.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:45 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:24:46.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:46 smithi066 ceph-mon[33137]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:46.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:46 smithi136 ceph-mon[33038]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:46.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:46 smithi031 ceph-mon[29457]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:48.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:48 smithi066 ceph-mon[33137]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:48.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:48 smithi136 ceph-mon[33038]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:48.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:48 smithi031 ceph-mon[29457]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:50.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:50 smithi066 ceph-mon[33137]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:50.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:50 smithi136 ceph-mon[33038]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:50.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:50 smithi031 ceph-mon[29457]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:52.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:52 smithi066 ceph-mon[33137]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:52.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:52 smithi136 ceph-mon[33038]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:52.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:52 smithi031 ceph-mon[29457]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:54 smithi066 ceph-mon[33137]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:54.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:54 smithi136 ceph-mon[33038]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:54.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:54 smithi031 ceph-mon[29457]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:56.909 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:56 smithi066 ceph-mon[33137]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:56.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:56 smithi136 ceph-mon[33038]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:56.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:56 smithi031 ceph-mon[29457]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:58.786 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:24:58 smithi031 ceph-mon[29457]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:58.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:24:58 smithi066 ceph-mon[33137]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:58.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:24:58 smithi136 ceph-mon[33038]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:00.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:00 smithi066 ceph-mon[33137]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:00.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:00 smithi136 ceph-mon[33038]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:00.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:00 smithi031 ceph-mon[29457]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:02.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:02 smithi066 ceph-mon[33137]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:25:02.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:02 smithi136 ceph-mon[33038]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:25:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:02 smithi031 ceph-mon[29457]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:25:04.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:04 smithi066 ceph-mon[33137]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:25:04.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:04 smithi136 ceph-mon[33038]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:25:04.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:04 smithi031 ceph-mon[29457]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:25:06.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:06 smithi066 ceph-mon[33137]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:25:06.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:06 smithi136 ceph-mon[33038]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:25:06.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:06 smithi031 ceph-mon[29457]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:25:08.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:08 smithi066 ceph-mon[33137]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:25:08.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:08 smithi136 ceph-mon[33038]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:25:08.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:08 smithi031 ceph-mon[29457]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:25:10.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:10 smithi066 ceph-mon[33137]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:25:10.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:10 smithi136 ceph-mon[33038]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:25:10.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:10 smithi031 ceph-mon[29457]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:25:12.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:12 smithi066 ceph-mon[33137]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:25:12.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:12 smithi136 ceph-mon[33038]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:25:12.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:12 smithi031 ceph-mon[29457]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:25:14.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:14 smithi066 ceph-mon[33137]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:25:14.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:14 smithi136 ceph-mon[33038]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:25:14.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:14 smithi031 ceph-mon[29457]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:25:16.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:16 smithi066 ceph-mon[33137]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:25:16.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:16 smithi136 ceph-mon[33038]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:25:16.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:16 smithi031 ceph-mon[29457]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T13:25:18.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:18 smithi066 ceph-mon[33137]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:25:18.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:18 smithi136 ceph-mon[33038]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:25:18.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:18 smithi031 ceph-mon[29457]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T13:25:20.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:20 smithi066 ceph-mon[33137]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:25:20.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:20 smithi136 ceph-mon[33038]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:25:20.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:20 smithi031 ceph-mon[29457]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:25:22.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:22 smithi066 ceph-mon[33137]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:25:22.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:22 smithi136 ceph-mon[33038]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:25:22.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:22 smithi031 ceph-mon[29457]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:25:24.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:24 smithi066 ceph-mon[33137]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:24.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:24 smithi136 ceph-mon[33038]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:24.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:24 smithi031 ceph-mon[29457]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:26.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:26 smithi066 ceph-mon[33137]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:26.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:26 smithi136 ceph-mon[33038]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:26.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:26 smithi031 ceph-mon[29457]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:28.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:28 smithi066 ceph-mon[33137]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:28.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:28 smithi136 ceph-mon[33038]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:28.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:28 smithi031 ceph-mon[29457]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:30.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:30 smithi066 ceph-mon[33137]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:30.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:30 smithi136 ceph-mon[33038]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:30.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:30 smithi031 ceph-mon[29457]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:32.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:32 smithi066 ceph-mon[33137]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:32.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:32 smithi136 ceph-mon[33038]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:32.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:32 smithi031 ceph-mon[29457]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:34.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:34 smithi066 ceph-mon[33137]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:34.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:34 smithi136 ceph-mon[33038]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:34.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:34 smithi031 ceph-mon[29457]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:37.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:36 smithi066 ceph-mon[33137]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:36 smithi136 ceph-mon[33038]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:37.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:36 smithi031 ceph-mon[29457]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:39.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:38 smithi066 ceph-mon[33137]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:38 smithi136 ceph-mon[33038]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:39.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:38 smithi031 ceph-mon[29457]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:41.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:40 smithi066 ceph-mon[33137]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:40 smithi136 ceph-mon[33038]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:41.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:40 smithi031 ceph-mon[29457]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:43.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:42 smithi066 ceph-mon[33137]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:42 smithi136 ceph-mon[33038]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:43.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:42 smithi031 ceph-mon[29457]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:45.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:44 smithi066 ceph-mon[33137]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:45.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:44 smithi136 ceph-mon[33038]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:45.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:44 smithi031 ceph-mon[29457]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:46.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:45 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:25:46.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:45 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:25:46.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:45 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:25:47.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:46 smithi066 ceph-mon[33137]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:47.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:46 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:25:47.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:46 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:25:47.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:46 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:25:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:46 smithi136 ceph-mon[33038]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:46 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:25:47.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:46 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:25:47.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:46 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:25:47.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:46 smithi031 ceph-mon[29457]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:47.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:46 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:25:47.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:46 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:25:47.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:46 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:25:49.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:48 smithi066 ceph-mon[33137]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:49.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:48 smithi136 ceph-mon[33038]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:49.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:48 smithi031 ceph-mon[29457]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:51.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:50 smithi066 ceph-mon[33137]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:51.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:50 smithi136 ceph-mon[33038]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:51.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:50 smithi031 ceph-mon[29457]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:53.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:52 smithi066 ceph-mon[33137]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:53.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:52 smithi136 ceph-mon[33038]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:53.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:52 smithi031 ceph-mon[29457]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:55.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:54 smithi066 ceph-mon[33137]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:55.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:54 smithi136 ceph-mon[33038]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:55.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:54 smithi031 ceph-mon[29457]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:56 smithi066 ceph-mon[33137]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:56 smithi136 ceph-mon[33038]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:57.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:56 smithi031 ceph-mon[29457]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:25:58 smithi066 ceph-mon[33137]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:25:58 smithi136 ceph-mon[33038]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:59.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:25:58 smithi031 ceph-mon[29457]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:01.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:00 smithi066 ceph-mon[33137]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:01.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:00 smithi136 ceph-mon[33038]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:01.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:00 smithi031 ceph-mon[29457]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:02 smithi066 ceph-mon[33137]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:26:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:02 smithi136 ceph-mon[33038]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:26:03.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:02 smithi031 ceph-mon[29457]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:26:05.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:04 smithi066 ceph-mon[33137]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:26:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:04 smithi136 ceph-mon[33038]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:26:05.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:04 smithi031 ceph-mon[29457]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:26:07.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:06 smithi066 ceph-mon[33137]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:26:07.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:06 smithi136 ceph-mon[33038]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:26:07.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:06 smithi031 ceph-mon[29457]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:26:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:08 smithi066 ceph-mon[33137]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:26:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:08 smithi066 ceph-mon[33137]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:26:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:08 smithi136 ceph-mon[33038]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:26:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:08 smithi136 ceph-mon[33038]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:26:09.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:08 smithi031 ceph-mon[29457]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:26:09.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:08 smithi031 ceph-mon[29457]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:26:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:10 smithi136 ceph-mon[33038]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:26:11.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:10 smithi031 ceph-mon[29457]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:26:11.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:10 smithi066 ceph-mon[33137]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:26:13.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:12 smithi136 ceph-mon[33038]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:26:13.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:12 smithi031 ceph-mon[29457]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:26:13.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:12 smithi066 ceph-mon[33137]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:26:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:14 smithi136 ceph-mon[33038]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:26:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:14 smithi136 ceph-mon[33038]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:26:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:14 smithi031 ceph-mon[29457]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:26:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:14 smithi031 ceph-mon[29457]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:26:15.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:14 smithi066 ceph-mon[33137]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:26:15.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:14 smithi066 ceph-mon[33137]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:26:17.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:16 smithi136 ceph-mon[33038]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:26:17.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:16 smithi031 ceph-mon[29457]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:26:17.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:16 smithi066 ceph-mon[33137]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:26:19.210 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:18 smithi066 ceph-mon[33137]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 215 B/s wr, 2 op/s 2024-04-25T13:26:19.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:18 smithi136 ceph-mon[33038]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 215 B/s wr, 2 op/s 2024-04-25T13:26:19.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:18 smithi031 ceph-mon[29457]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 215 B/s wr, 2 op/s 2024-04-25T13:26:21.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:20 smithi136 ceph-mon[33038]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:26:21.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:20 smithi031 ceph-mon[29457]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:26:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:20 smithi066 ceph-mon[33137]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:26:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:22 smithi136 ceph-mon[33038]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:26:23.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:22 smithi031 ceph-mon[29457]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:26:23.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:22 smithi066 ceph-mon[33137]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:26:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:24 smithi136 ceph-mon[33038]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:26:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:24 smithi031 ceph-mon[29457]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:26:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:24 smithi066 ceph-mon[33137]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:26:27.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:26 smithi136 ceph-mon[33038]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:26:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:26 smithi031 ceph-mon[29457]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:26:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:26 smithi066 ceph-mon[33137]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:26:29.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:28 smithi136 ceph-mon[33038]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:26:29.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:28 smithi031 ceph-mon[29457]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:26:29.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:28 smithi066 ceph-mon[33137]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:26:31.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:30 smithi136 ceph-mon[33038]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:31.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:30 smithi031 ceph-mon[29457]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:30 smithi066 ceph-mon[33137]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:32 smithi136 ceph-mon[33038]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:33.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:32 smithi031 ceph-mon[29457]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:32 smithi066 ceph-mon[33137]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:35.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:34 smithi136 ceph-mon[33038]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:35.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:34 smithi031 ceph-mon[29457]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:34 smithi066 ceph-mon[33137]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:36 smithi136 ceph-mon[33038]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:37.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:36 smithi031 ceph-mon[29457]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:37.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:36 smithi066 ceph-mon[33137]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:38 smithi136 ceph-mon[33038]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:39.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:38 smithi031 ceph-mon[29457]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:39.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:38 smithi066 ceph-mon[33137]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:40 smithi136 ceph-mon[33038]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:41.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:40 smithi031 ceph-mon[29457]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:41.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:40 smithi066 ceph-mon[33137]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:42 smithi136 ceph-mon[33038]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:42 smithi031 ceph-mon[29457]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:43.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:42 smithi066 ceph-mon[33137]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:45.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:44 smithi136 ceph-mon[33038]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:45.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:44 smithi031 ceph-mon[29457]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:45.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:44 smithi066 ceph-mon[33137]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:46 smithi136 ceph-mon[33038]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:46 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:26:47.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:46 smithi031 ceph-mon[29457]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:47.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:46 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:26:47.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:46 smithi066 ceph-mon[33137]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:47.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:46 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:26:48.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:47 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:26:48.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:47 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:26:48.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:47 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:26:48.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:47 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:26:48.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:47 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:26:48.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:47 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:26:48.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:47 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:26:48.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:47 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:26:48.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:47 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:26:49.309 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:49 smithi066 ceph-mon[33137]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:49 smithi136 ceph-mon[33038]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:49 smithi031 ceph-mon[29457]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:51 smithi066 ceph-mon[33137]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:51.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:51 smithi136 ceph-mon[33038]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:51.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:51 smithi031 ceph-mon[29457]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:53 smithi066 ceph-mon[33137]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:53.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:53 smithi136 ceph-mon[33038]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:53.486 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:53 smithi031 ceph-mon[29457]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:55.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:55 smithi066 ceph-mon[33137]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:55.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:55 smithi136 ceph-mon[33038]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:55.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:55 smithi031 ceph-mon[29457]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:57.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:57 smithi066 ceph-mon[33137]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:57.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:57 smithi136 ceph-mon[33038]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:57.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:57 smithi031 ceph-mon[29457]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:59.286 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:26:59 smithi031 ceph-mon[29457]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:59.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:26:59 smithi066 ceph-mon[33137]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:59.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:26:59 smithi136 ceph-mon[33038]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:01.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:01 smithi066 ceph-mon[33137]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:01 smithi136 ceph-mon[33038]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:01.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:01 smithi031 ceph-mon[29457]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:03.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:03 smithi066 ceph-mon[33137]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:27:03.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:03 smithi136 ceph-mon[33038]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:27:03.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:03 smithi031 ceph-mon[29457]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:27:05.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:05 smithi066 ceph-mon[33137]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:27:05.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:05 smithi136 ceph-mon[33038]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:27:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:05 smithi031 ceph-mon[29457]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:27:07.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:07 smithi066 ceph-mon[33137]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:27:07.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:07 smithi136 ceph-mon[33038]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:27:07.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:07 smithi031 ceph-mon[29457]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:27:09.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:09 smithi066 ceph-mon[33137]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:27:09.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:09 smithi066 ceph-mon[33137]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:27:09.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:09 smithi136 ceph-mon[33038]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:27:09.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:09 smithi136 ceph-mon[33038]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:27:09.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:09 smithi031 ceph-mon[29457]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:27:09.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:09 smithi031 ceph-mon[29457]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:27:11.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:11 smithi066 ceph-mon[33137]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:27:11.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:11 smithi136 ceph-mon[33038]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:27:11.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:11 smithi031 ceph-mon[29457]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:27:13.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:13 smithi066 ceph-mon[33137]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:27:13.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:13 smithi136 ceph-mon[33038]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:27:13.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:13 smithi031 ceph-mon[29457]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:27:15.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:15 smithi066 ceph-mon[33137]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:27:15.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:15 smithi066 ceph-mon[33137]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:27:15.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:15 smithi136 ceph-mon[33038]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:27:15.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:15 smithi136 ceph-mon[33038]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:27:15.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:15 smithi031 ceph-mon[29457]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:27:15.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:15 smithi031 ceph-mon[29457]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:27:17.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:17 smithi066 ceph-mon[33137]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:27:17.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:17 smithi136 ceph-mon[33038]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:27:17.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:17 smithi031 ceph-mon[29457]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:27:19.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:19 smithi066 ceph-mon[33137]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 218 B/s wr, 2 op/s 2024-04-25T13:27:19.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:19 smithi136 ceph-mon[33038]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 218 B/s wr, 2 op/s 2024-04-25T13:27:19.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:19 smithi031 ceph-mon[29457]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 218 B/s wr, 2 op/s 2024-04-25T13:27:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:21 smithi066 ceph-mon[33137]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:27:21.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:21 smithi136 ceph-mon[33038]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:27:21.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:21 smithi031 ceph-mon[29457]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:27:23.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:23 smithi066 ceph-mon[33137]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:27:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:23 smithi136 ceph-mon[33038]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:27:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:23 smithi031 ceph-mon[29457]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:27:25.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:25 smithi066 ceph-mon[33137]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:27:25.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:25 smithi136 ceph-mon[33038]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:27:25.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:25 smithi031 ceph-mon[29457]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:27:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:27 smithi066 ceph-mon[33137]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:27:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:27 smithi136 ceph-mon[33038]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:27:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:27 smithi031 ceph-mon[29457]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:27:29.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:29 smithi066 ceph-mon[33137]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:27:29.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:29 smithi136 ceph-mon[33038]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:27:29.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:29 smithi031 ceph-mon[29457]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:27:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:31 smithi066 ceph-mon[33137]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:31 smithi136 ceph-mon[33038]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:31 smithi031 ceph-mon[29457]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:33 smithi066 ceph-mon[33137]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:33 smithi136 ceph-mon[33038]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:33 smithi031 ceph-mon[29457]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:35 smithi066 ceph-mon[33137]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:35.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:35 smithi136 ceph-mon[33038]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:35.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:35 smithi031 ceph-mon[29457]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:37.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:37 smithi066 ceph-mon[33137]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:37.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:37 smithi136 ceph-mon[33038]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:37.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:37 smithi031 ceph-mon[29457]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:39.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:39 smithi066 ceph-mon[33137]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:39.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:39 smithi136 ceph-mon[33038]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:39.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:39 smithi031 ceph-mon[29457]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:41.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:41 smithi066 ceph-mon[33137]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:41.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:41 smithi136 ceph-mon[33038]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:41.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:41 smithi031 ceph-mon[29457]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:43.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:43 smithi066 ceph-mon[33137]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:43.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:43 smithi136 ceph-mon[33038]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:43.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:43 smithi031 ceph-mon[29457]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:45.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:45 smithi066 ceph-mon[33137]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:45.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:45 smithi136 ceph-mon[33038]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:45.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:45 smithi031 ceph-mon[29457]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:47.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:47 smithi066 ceph-mon[33137]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:47.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:47 smithi136 ceph-mon[33038]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:47.483 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:47 smithi031 ceph-mon[29457]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:48.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:48 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:27:48.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:48 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:27:48.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:48 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:27:48.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:48 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:27:48.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:48 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:27:48.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:48 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:27:48.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:48 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:27:48.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:48 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:27:48.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:48 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:27:48.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:48 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:27:48.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:48 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:27:48.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:48 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:27:49.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:49 smithi066 ceph-mon[33137]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:49 smithi136 ceph-mon[33038]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:49 smithi031 ceph-mon[29457]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:50.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:50 smithi066 ceph-mon[33137]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:50.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:50 smithi136 ceph-mon[33038]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:50.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:50 smithi031 ceph-mon[29457]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:52.734 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:52 smithi031 ceph-mon[29457]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:52.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:52 smithi066 ceph-mon[33137]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:52.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:52 smithi136 ceph-mon[33038]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:54 smithi066 ceph-mon[33137]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:54.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:54 smithi136 ceph-mon[33038]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:54.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:54 smithi031 ceph-mon[29457]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:57.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:56 smithi066 ceph-mon[33137]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:56 smithi136 ceph-mon[33038]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:57.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:56 smithi031 ceph-mon[29457]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:27:58 smithi066 ceph-mon[33137]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:27:58 smithi136 ceph-mon[33038]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:59.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:27:58 smithi031 ceph-mon[29457]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:01.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:00 smithi066 ceph-mon[33137]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:01.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:00 smithi136 ceph-mon[33038]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:01.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:00 smithi031 ceph-mon[29457]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:03.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:02 smithi066 ceph-mon[33137]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:28:03.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:02 smithi136 ceph-mon[33038]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:28:03.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:02 smithi031 ceph-mon[29457]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:28:05.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:04 smithi066 ceph-mon[33137]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:28:05.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:04 smithi136 ceph-mon[33038]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:28:05.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:04 smithi031 ceph-mon[29457]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:28:07.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:06 smithi066 ceph-mon[33137]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:28:07.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:06 smithi136 ceph-mon[33038]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:28:07.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:06 smithi031 ceph-mon[29457]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:28:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:08 smithi066 ceph-mon[33137]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:28:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:08 smithi066 ceph-mon[33137]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:28:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:08 smithi136 ceph-mon[33038]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:28:09.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:08 smithi136 ceph-mon[33038]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:28:09.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:08 smithi031 ceph-mon[29457]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:28:09.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:08 smithi031 ceph-mon[29457]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:28:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:10 smithi066 ceph-mon[33137]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:28:11.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:10 smithi136 ceph-mon[33038]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:28:11.233 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:10 smithi031 ceph-mon[29457]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:28:13.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:12 smithi136 ceph-mon[33038]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:28:13.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:12 smithi031 ceph-mon[29457]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:28:13.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:12 smithi066 ceph-mon[33137]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:28:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:14 smithi136 ceph-mon[33038]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:28:15.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:14 smithi136 ceph-mon[33038]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T13:28:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:14 smithi031 ceph-mon[29457]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:28:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:14 smithi031 ceph-mon[29457]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T13:28:15.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:14 smithi066 ceph-mon[33137]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:28:15.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:14 smithi066 ceph-mon[33137]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T13:28:17.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:16 smithi136 ceph-mon[33038]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:28:17.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:16 smithi031 ceph-mon[29457]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:28:17.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:16 smithi066 ceph-mon[33137]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:28:19.211 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:18 smithi066 ceph-mon[33137]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-25T13:28:19.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:18 smithi136 ceph-mon[33038]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-25T13:28:19.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:18 smithi031 ceph-mon[29457]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-25T13:28:21.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:20 smithi136 ceph-mon[33038]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:28:21.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:20 smithi031 ceph-mon[29457]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:28:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:20 smithi066 ceph-mon[33137]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:28:23.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:22 smithi136 ceph-mon[33038]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:28:23.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:22 smithi031 ceph-mon[29457]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:28:23.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:22 smithi066 ceph-mon[33137]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:28:25.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:24 smithi136 ceph-mon[33038]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:28:25.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:24 smithi031 ceph-mon[29457]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:28:25.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:24 smithi066 ceph-mon[33137]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:28:27.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:26 smithi136 ceph-mon[33038]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:28:27.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:26 smithi031 ceph-mon[29457]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:28:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:26 smithi066 ceph-mon[33137]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:28:29.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:28 smithi136 ceph-mon[33038]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:28:29.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:28 smithi031 ceph-mon[29457]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:28:29.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:28 smithi066 ceph-mon[33137]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:28:31.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:30 smithi136 ceph-mon[33038]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:31.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:30 smithi031 ceph-mon[29457]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:30 smithi066 ceph-mon[33137]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:33.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:32 smithi136 ceph-mon[33038]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:33.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:32 smithi031 ceph-mon[29457]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:32 smithi066 ceph-mon[33137]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:35.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:34 smithi136 ceph-mon[33038]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:35.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:34 smithi031 ceph-mon[29457]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:34 smithi066 ceph-mon[33137]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:37.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:36 smithi136 ceph-mon[33038]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:37.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:36 smithi031 ceph-mon[29457]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:37.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:36 smithi066 ceph-mon[33137]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:39.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:38 smithi136 ceph-mon[33038]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:39.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:38 smithi031 ceph-mon[29457]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:39.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:38 smithi066 ceph-mon[33137]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:41.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:40 smithi136 ceph-mon[33038]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:41.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:40 smithi031 ceph-mon[29457]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:41.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:40 smithi066 ceph-mon[33137]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:43.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:42 smithi136 ceph-mon[33038]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:42 smithi031 ceph-mon[29457]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:43.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:42 smithi066 ceph-mon[33137]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:45.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:44 smithi136 ceph-mon[33038]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:45.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:44 smithi031 ceph-mon[29457]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:45.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:44 smithi066 ceph-mon[33137]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:47.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:46 smithi136 ceph-mon[33038]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:47.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:46 smithi031 ceph-mon[29457]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:47.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:46 smithi066 ceph-mon[33137]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:48.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:48 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:28:48.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:48 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:28:48.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:48 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:28:49.310 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:49 smithi066 ceph-mon[33137]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:49.310 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:49 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:28:49.310 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:49 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:28:49.310 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:49 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:28:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:49 smithi136 ceph-mon[33038]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:49 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:28:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:49 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:28:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:49 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:28:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:49 smithi031 ceph-mon[29457]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:49 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:28:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:49 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:28:49.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:49 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:28:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:51 smithi066 ceph-mon[33137]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:51.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:51 smithi136 ceph-mon[33038]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:51.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:51 smithi031 ceph-mon[29457]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:53.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:53 smithi066 ceph-mon[33137]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:53.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:53 smithi136 ceph-mon[33038]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:53.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:53 smithi031 ceph-mon[29457]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:55.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:55 smithi066 ceph-mon[33137]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:55.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:55 smithi136 ceph-mon[33038]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:55.485 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:55 smithi031 ceph-mon[29457]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:57.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:57 smithi066 ceph-mon[33137]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:57.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:57 smithi136 ceph-mon[33038]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:57.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:57 smithi031 ceph-mon[29457]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:59.286 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:28:59 smithi031 ceph-mon[29457]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:59.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:28:59 smithi066 ceph-mon[33137]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:59.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:28:59 smithi136 ceph-mon[33038]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:01.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:01 smithi066 ceph-mon[33137]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:01.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:01 smithi136 ceph-mon[33038]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:01.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:01 smithi031 ceph-mon[29457]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:03.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:03 smithi066 ceph-mon[33137]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:29:03.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:03 smithi136 ceph-mon[33038]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:29:03.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:03 smithi031 ceph-mon[29457]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:29:05.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:05 smithi066 ceph-mon[33137]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:29:05.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:05 smithi136 ceph-mon[33038]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:29:05.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:05 smithi031 ceph-mon[29457]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:29:07.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:07 smithi066 ceph-mon[33137]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:29:07.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:07 smithi136 ceph-mon[33038]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:29:07.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:07 smithi031 ceph-mon[29457]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:29:09.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:09 smithi066 ceph-mon[33137]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:29:09.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:09 smithi066 ceph-mon[33137]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T13:29:09.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:09 smithi136 ceph-mon[33038]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:29:09.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:09 smithi136 ceph-mon[33038]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T13:29:09.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:09 smithi031 ceph-mon[29457]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:29:09.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:09 smithi031 ceph-mon[29457]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T13:29:11.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:11 smithi066 ceph-mon[33137]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:29:11.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:11 smithi136 ceph-mon[33038]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:29:11.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:11 smithi031 ceph-mon[29457]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:29:13.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:13 smithi066 ceph-mon[33137]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:29:13.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:13 smithi136 ceph-mon[33038]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:29:13.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:13 smithi031 ceph-mon[29457]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:29:15.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:15 smithi066 ceph-mon[33137]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:29:15.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:15 smithi066 ceph-mon[33137]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T13:29:15.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:15 smithi136 ceph-mon[33038]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:29:15.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:15 smithi136 ceph-mon[33038]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T13:29:15.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:15 smithi031 ceph-mon[29457]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:29:15.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:15 smithi031 ceph-mon[29457]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T13:29:17.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:17 smithi066 ceph-mon[33137]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:29:17.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:17 smithi136 ceph-mon[33038]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:29:17.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:17 smithi031 ceph-mon[29457]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:29:19.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:19 smithi066 ceph-mon[33137]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 217 B/s wr, 2 op/s 2024-04-25T13:29:19.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:19 smithi136 ceph-mon[33038]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 217 B/s wr, 2 op/s 2024-04-25T13:29:19.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:19 smithi031 ceph-mon[29457]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 217 B/s wr, 2 op/s 2024-04-25T13:29:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:21 smithi066 ceph-mon[33137]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:29:21.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:21 smithi136 ceph-mon[33038]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:29:21.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:21 smithi031 ceph-mon[29457]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:29:23.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:23 smithi066 ceph-mon[33137]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:29:23.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:23 smithi136 ceph-mon[33038]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:29:23.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:23 smithi031 ceph-mon[29457]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:29:25.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:25 smithi066 ceph-mon[33137]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:29:25.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:25 smithi136 ceph-mon[33038]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:29:25.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:25 smithi031 ceph-mon[29457]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:29:27.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:27 smithi066 ceph-mon[33137]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:29:27.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:27 smithi136 ceph-mon[33038]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:29:27.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:27 smithi031 ceph-mon[29457]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 97 B/s wr, 2 op/s 2024-04-25T13:29:29.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:29 smithi066 ceph-mon[33137]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:29:29.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:29 smithi136 ceph-mon[33038]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:29:29.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:29 smithi031 ceph-mon[29457]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:29:31.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:31 smithi066 ceph-mon[33137]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:31.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:31 smithi136 ceph-mon[33038]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:31.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:31 smithi031 ceph-mon[29457]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:33.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:33 smithi066 ceph-mon[33137]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:33.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:33 smithi136 ceph-mon[33038]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:33.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:33 smithi031 ceph-mon[29457]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:35.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:35 smithi066 ceph-mon[33137]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:35.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:35 smithi136 ceph-mon[33038]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:35.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:35 smithi031 ceph-mon[29457]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:37.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:37 smithi066 ceph-mon[33137]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:37.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:37 smithi136 ceph-mon[33038]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:37.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:37 smithi031 ceph-mon[29457]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:38.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T13:29:38.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.151+0000 7fe90d744640 1 -- 172.21.15.31:0/1828648141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe908107290 msgr2=0x7fe90810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.151+0000 7fe90d744640 1 --2- 172.21.15.31:0/1828648141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe908107290 0x7fe90810ee60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe900009ca0 tx=0x7fe90002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.151+0000 7fe90d744640 1 -- 172.21.15.31:0/1828648141 shutdown_connections 2024-04-25T13:29:38.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.151+0000 7fe90d744640 1 --2- 172.21.15.31:0/1828648141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe908107290 0x7fe90810ee60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.151+0000 7fe90d744640 1 --2- 172.21.15.31:0/1828648141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe908102b70 0x7fe908106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.151+0000 7fe90d744640 1 --2- 172.21.15.31:0/1828648141 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9081021c0 0x7fe9081025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.151+0000 7fe90d744640 1 -- 172.21.15.31:0/1828648141 >> 172.21.15.31:0/1828648141 conn(0x7fe9080fda20 msgr2=0x7fe9080ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:38.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.151+0000 7fe90d744640 1 -- 172.21.15.31:0/1828648141 shutdown_connections 2024-04-25T13:29:38.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.151+0000 7fe90d744640 1 -- 172.21.15.31:0/1828648141 wait complete. 2024-04-25T13:29:38.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.152+0000 7fe90d744640 1 Processor -- start 2024-04-25T13:29:38.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.152+0000 7fe90d744640 1 -- start start 2024-04-25T13:29:38.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.152+0000 7fe90d744640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9081021c0 0x7fe908074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.152+0000 7fe90d744640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe908107290 0x7fe908074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.152+0000 7fe90d744640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe908074cd0 0x7fe9081a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.152+0000 7fe90d744640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9081121e0 con 0x7fe9081021c0 2024-04-25T13:29:38.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.152+0000 7fe90d744640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe908112060 con 0x7fe908074cd0 2024-04-25T13:29:38.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.152+0000 7fe90d744640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe908112360 con 0x7fe908107290 2024-04-25T13:29:38.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.152+0000 7fe906ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9081021c0 0x7fe908074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.152+0000 7fe906ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9081021c0 0x7fe908074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41580/0 (socket says 172.21.15.31:41580) 2024-04-25T13:29:38.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.152+0000 7fe906ffd640 1 -- 172.21.15.31:0/537822050 learned_addr learned my addr 172.21.15.31:0/537822050 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:29:38.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe9077fe640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe908074cd0 0x7fe9081a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe9067fc640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe908107290 0x7fe908074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe906ffd640 1 -- 172.21.15.31:0/537822050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe908107290 msgr2=0x7fe908074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe906ffd640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe908107290 0x7fe908074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe906ffd640 1 -- 172.21.15.31:0/537822050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe908074cd0 msgr2=0x7fe9081a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe906ffd640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe908074cd0 0x7fe9081a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe906ffd640 1 -- 172.21.15.31:0/537822050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9081a6f10 con 0x7fe9081021c0 2024-04-25T13:29:38.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe906ffd640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9081021c0 0x7fe908074250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800d950 tx=0x7fe8f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:38.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe8effff640 1 -- 172.21.15.31:0/537822050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8017070 con 0x7fe9081021c0 2024-04-25T13:29:38.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe8effff640 1 -- 172.21.15.31:0/537822050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8f8009c30 con 0x7fe9081021c0 2024-04-25T13:29:38.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe90d744640 1 -- 172.21.15.31:0/537822050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9081a7200 con 0x7fe9081021c0 2024-04-25T13:29:38.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe8effff640 1 -- 172.21.15.31:0/537822050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800f620 con 0x7fe9081021c0 2024-04-25T13:29:38.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.153+0000 7fe90d744640 1 -- 172.21.15.31:0/537822050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9081a7750 con 0x7fe9081021c0 2024-04-25T13:29:38.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.154+0000 7fe90d744640 1 -- 172.21.15.31:0/537822050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90804fce0 con 0x7fe9081021c0 2024-04-25T13:29:38.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.155+0000 7fe8effff640 1 -- 172.21.15.31:0/537822050 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe8f8010460 con 0x7fe9081021c0 2024-04-25T13:29:38.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.156+0000 7fe8effff640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8f00a3dc0 0x7fe8f00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.156+0000 7fe9067fc640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8f00a3dc0 0x7fe8f00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.156+0000 7fe8effff640 1 -- 172.21.15.31:0/537822050 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fe8f80d0460 con 0x7fe9081021c0 2024-04-25T13:29:38.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.157+0000 7fe9067fc640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8f00a3dc0 0x7fe8f00a6280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe908075c90 tx=0x7fe8fc00b390 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:38.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.161+0000 7fe8effff640 1 -- 172.21.15.31:0/537822050 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8f8009de0 con 0x7fe9081021c0 2024-04-25T13:29:38.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.374+0000 7fe90d744640 1 -- 172.21.15.31:0/537822050 --> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fe90810a530 con 0x7fe8f00a3dc0 2024-04-25T13:29:38.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.382+0000 7fe8effff640 1 -- 172.21.15.31:0/537822050 <== mgr.24922 v2:172.21.15.31:6800/332946768 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fe8fc010b40 con 0x7fe8f00a3dc0 2024-04-25T13:29:38.384 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T13:19:36", "first": "2024-04-25T13:20:00", "last": "2024-04-25T13:29:00", "last_pruned": "2024-04-25T13:29:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T13:29:38.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 -- 172.21.15.31:0/537822050 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8f00a3dc0 msgr2=0x7fe8f00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8f00a3dc0 0x7fe8f00a6280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe908075c90 tx=0x7fe8fc00b390 comp rx=0 tx=0).stop 2024-04-25T13:29:38.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 -- 172.21.15.31:0/537822050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9081021c0 msgr2=0x7fe908074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9081021c0 0x7fe908074250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800d950 tx=0x7fe8f800de10 comp rx=0 tx=0).stop 2024-04-25T13:29:38.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 -- 172.21.15.31:0/537822050 shutdown_connections 2024-04-25T13:29:38.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8f00a3dc0 0x7fe8f00a6280 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe908074cd0 0x7fe9081a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe908107290 0x7fe908074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 --2- 172.21.15.31:0/537822050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9081021c0 0x7fe908074250 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 -- 172.21.15.31:0/537822050 >> 172.21.15.31:0/537822050 conn(0x7fe9080fda20 msgr2=0x7fe9080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:38.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 -- 172.21.15.31:0/537822050 shutdown_connections 2024-04-25T13:29:38.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.384+0000 7fe90d744640 1 -- 172.21.15.31:0/537822050 wait complete. 2024-04-25T13:29:38.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T13:29:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.500+0000 7faa5de79640 1 -- 172.21.15.31:0/1351169223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa581021c0 msgr2=0x7faa581025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.500+0000 7faa5de79640 1 --2- 172.21.15.31:0/1351169223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa581021c0 0x7faa581025a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faa48009900 tx=0x7faa4802a410 comp rx=0 tx=0).stop 2024-04-25T13:29:38.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.500+0000 7faa5de79640 1 -- 172.21.15.31:0/1351169223 shutdown_connections 2024-04-25T13:29:38.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.500+0000 7faa5de79640 1 --2- 172.21.15.31:0/1351169223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa58107290 0x7faa5810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.500+0000 7faa5de79640 1 --2- 172.21.15.31:0/1351169223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa58102b70 0x7faa58106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.500+0000 7faa5de79640 1 --2- 172.21.15.31:0/1351169223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa581021c0 0x7faa581025a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.500+0000 7faa5de79640 1 -- 172.21.15.31:0/1351169223 >> 172.21.15.31:0/1351169223 conn(0x7faa580fda20 msgr2=0x7faa580ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:38.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.500+0000 7faa5de79640 1 -- 172.21.15.31:0/1351169223 shutdown_connections 2024-04-25T13:29:38.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.500+0000 7faa5de79640 1 -- 172.21.15.31:0/1351169223 wait complete. 2024-04-25T13:29:38.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa5de79640 1 Processor -- start 2024-04-25T13:29:38.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa5de79640 1 -- start start 2024-04-25T13:29:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa5de79640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa581021c0 0x7faa58198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa5de79640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa58107290 0x7faa58198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa5de79640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa5819cfe0 0x7faa5819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa5de79640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faa581121f0 con 0x7faa581021c0 2024-04-25T13:29:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa5de79640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa58112070 con 0x7faa58107290 2024-04-25T13:29:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa5de79640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa58112370 con 0x7faa5819cfe0 2024-04-25T13:29:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa581021c0 0x7faa58198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa581021c0 0x7faa58198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41600/0 (socket says 172.21.15.31:41600) 2024-04-25T13:29:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa577fe640 1 -- 172.21.15.31:0/1087599948 learned_addr learned my addr 172.21.15.31:0/1087599948 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:29:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa56ffd640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa58107290 0x7faa58198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa57fff640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa5819cfe0 0x7faa5819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa577fe640 1 -- 172.21.15.31:0/1087599948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa5819cfe0 msgr2=0x7faa5819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa577fe640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa5819cfe0 0x7faa5819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa577fe640 1 -- 172.21.15.31:0/1087599948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa58107290 msgr2=0x7faa58198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa577fe640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa58107290 0x7faa58198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.501+0000 7faa577fe640 1 -- 172.21.15.31:0/1087599948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5819dad0 con 0x7faa581021c0 2024-04-25T13:29:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.502+0000 7faa577fe640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa581021c0 0x7faa58198770 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faa48005940 tx=0x7faa480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.502+0000 7faa54ff9640 1 -- 172.21.15.31:0/1087599948 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4803f030 con 0x7faa581021c0 2024-04-25T13:29:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.502+0000 7faa5de79640 1 -- 172.21.15.31:0/1087599948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5819dd60 con 0x7faa581021c0 2024-04-25T13:29:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.502+0000 7faa54ff9640 1 -- 172.21.15.31:0/1087599948 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa48003cb0 con 0x7faa581021c0 2024-04-25T13:29:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.502+0000 7faa54ff9640 1 -- 172.21.15.31:0/1087599948 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48035070 con 0x7faa581021c0 2024-04-25T13:29:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.502+0000 7faa5de79640 1 -- 172.21.15.31:0/1087599948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa581a5580 con 0x7faa581021c0 2024-04-25T13:29:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.503+0000 7faa5de79640 1 -- 172.21.15.31:0/1087599948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa5804fce0 con 0x7faa581021c0 2024-04-25T13:29:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.504+0000 7faa54ff9640 1 -- 172.21.15.31:0/1087599948 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa48042030 con 0x7faa581021c0 2024-04-25T13:29:38.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.505+0000 7faa54ff9640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7faa400a3dc0 0x7faa400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.505+0000 7faa56ffd640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7faa400a3dc0 0x7faa400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.505+0000 7faa54ff9640 1 -- 172.21.15.31:0/1087599948 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7faa480f1060 con 0x7faa581021c0 2024-04-25T13:29:38.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.505+0000 7faa56ffd640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7faa400a3dc0 0x7faa400a6280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faa581087a0 tx=0x7faa4c00a250 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:38.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.510+0000 7faa54ff9640 1 -- 172.21.15.31:0/1087599948 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa480bd040 con 0x7faa581021c0 2024-04-25T13:29:38.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.720+0000 7faa5de79640 1 -- 172.21.15.31:0/1087599948 --> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7faa5810a530 con 0x7faa400a3dc0 2024-04-25T13:29:38.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.727+0000 7faa54ff9640 1 -- 172.21.15.31:0/1087599948 <== mgr.24922 v2:172.21.15.31:6800/332946768 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7faa4c00a7b0 con 0x7faa400a3dc0 2024-04-25T13:29:38.728 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1m 6m3h 2024-04-25T13:29:38.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 -- 172.21.15.31:0/1087599948 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7faa400a3dc0 msgr2=0x7faa400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7faa400a3dc0 0x7faa400a6280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faa581087a0 tx=0x7faa4c00a250 comp rx=0 tx=0).stop 2024-04-25T13:29:38.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 -- 172.21.15.31:0/1087599948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa581021c0 msgr2=0x7faa58198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa581021c0 0x7faa58198770 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faa48005940 tx=0x7faa480042f0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 -- 172.21.15.31:0/1087599948 shutdown_connections 2024-04-25T13:29:38.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7faa400a3dc0 0x7faa400a6280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa5819cfe0 0x7faa5819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa58107290 0x7faa58198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 --2- 172.21.15.31:0/1087599948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa581021c0 0x7faa58198770 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 -- 172.21.15.31:0/1087599948 >> 172.21.15.31:0/1087599948 conn(0x7faa580fda20 msgr2=0x7faa580fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:38.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 -- 172.21.15.31:0/1087599948 shutdown_connections 2024-04-25T13:29:38.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.729+0000 7faa5de79640 1 -- 172.21.15.31:0/1087599948 wait complete. 2024-04-25T13:29:38.741 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T13:29:38.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:29:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.854+0000 7f948d7e5640 1 -- 172.21.15.31:0/2155568743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9488102240 msgr2=0x7f9488102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.854+0000 7f948d7e5640 1 --2- 172.21.15.31:0/2155568743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9488102240 0x7f9488102620 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9480009ca0 tx=0x7f948002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.854+0000 7f948d7e5640 1 -- 172.21.15.31:0/2155568743 shutdown_connections 2024-04-25T13:29:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.854+0000 7f948d7e5640 1 --2- 172.21.15.31:0/2155568743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9488107280 0x7f948810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.854+0000 7f948d7e5640 1 --2- 172.21.15.31:0/2155568743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102bf0 0x7f9488106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.854+0000 7f948d7e5640 1 --2- 172.21.15.31:0/2155568743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9488102240 0x7f9488102620 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.854+0000 7f948d7e5640 1 -- 172.21.15.31:0/2155568743 >> 172.21.15.31:0/2155568743 conn(0x7f94880790e0 msgr2=0x7f94880794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.854+0000 7f948d7e5640 1 -- 172.21.15.31:0/2155568743 shutdown_connections 2024-04-25T13:29:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.854+0000 7f948d7e5640 1 -- 172.21.15.31:0/2155568743 wait complete. 2024-04-25T13:29:38.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.854+0000 7f948d7e5640 1 Processor -- start 2024-04-25T13:29:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f948d7e5640 1 -- start start 2024-04-25T13:29:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f948d7e5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102240 0x7f948819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f948d7e5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9488107280 0x7f948819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f948d7e5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881985a0 0x7f9488198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f948d7e5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9488112260 con 0x7f9488107280 2024-04-25T13:29:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f948d7e5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f94881120e0 con 0x7f9488102240 2024-04-25T13:29:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f948d7e5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94881123e0 con 0x7f94881985a0 2024-04-25T13:29:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f94867fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9488107280 0x7f948819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f94867fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9488107280 0x7f948819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41610/0 (socket says 172.21.15.31:41610) 2024-04-25T13:29:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f94867fc640 1 -- 172.21.15.31:0/598762518 learned_addr learned my addr 172.21.15.31:0/598762518 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:29:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f9486ffd640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102240 0x7f948819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f94877fe640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881985a0 0x7f9488198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f94867fc640 1 -- 172.21.15.31:0/598762518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881985a0 msgr2=0x7f9488198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f94867fc640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881985a0 0x7f9488198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f94867fc640 1 -- 172.21.15.31:0/598762518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102240 msgr2=0x7f948819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f94867fc640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102240 0x7f948819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f94867fc640 1 -- 172.21.15.31:0/598762518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94881990c0 con 0x7f9488107280 2024-04-25T13:29:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f94877fe640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881985a0 0x7f9488198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:29:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.855+0000 7f94867fc640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9488107280 0x7f948819e930 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c970 tx=0x7f947c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.856+0000 7f946ffff640 1 -- 172.21.15.31:0/598762518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c007bb0 con 0x7f9488107280 2024-04-25T13:29:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.856+0000 7f946ffff640 1 -- 172.21.15.31:0/598762518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f947c007d50 con 0x7f9488107280 2024-04-25T13:29:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.856+0000 7f948d7e5640 1 -- 172.21.15.31:0/598762518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94881993b0 con 0x7f9488107280 2024-04-25T13:29:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.856+0000 7f946ffff640 1 -- 172.21.15.31:0/598762518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c00f450 con 0x7f9488107280 2024-04-25T13:29:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.856+0000 7f948d7e5640 1 -- 172.21.15.31:0/598762518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94881a5600 con 0x7f9488107280 2024-04-25T13:29:38.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.857+0000 7f948d7e5640 1 -- 172.21.15.31:0/598762518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94880691c0 con 0x7f9488107280 2024-04-25T13:29:38.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.858+0000 7f946ffff640 1 -- 172.21.15.31:0/598762518 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f947c004eb0 con 0x7f9488107280 2024-04-25T13:29:38.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.858+0000 7f946ffff640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f94700a3d20 0x7f94700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:38.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.859+0000 7f946ffff640 1 -- 172.21.15.31:0/598762518 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f947c0d0ac0 con 0x7f9488107280 2024-04-25T13:29:38.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.859+0000 7f9486ffd640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f94700a3d20 0x7f94700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:38.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.859+0000 7f9486ffd640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f94700a3d20 0x7f94700a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f948002f500 tx=0x7f948002f310 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:38.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:38.862+0000 7f946ffff640 1 -- 172.21.15.31:0/598762518 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f947c014030 con 0x7f9488107280 2024-04-25T13:29:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.049+0000 7f948d7e5640 1 -- 172.21.15.31:0/598762518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9488199ad0 con 0x7f9488107280 2024-04-25T13:29:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.051+0000 7f946ffff640 1 -- 172.21.15.31:0/598762518 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+23213 (secure 0 0 0) 0x7f9488199ad0 con 0x7f9488107280 2024-04-25T13:29:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:29:39.052 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":104,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:29:14.892035+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.2e","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2153915817":"2024-04-26T13:19:35.271090+0000","172.21.15.31:6801/958678691":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/1259986822":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/2022600806":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/958678691":"2024-04-26T13:19:35.271090+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/996243284":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.052+0000 7f948d7e5640 1 -- 172.21.15.31:0/598762518 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f94700a3d20 msgr2=0x7f94700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.052+0000 7f948d7e5640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f94700a3d20 0x7f94700a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f948002f500 tx=0x7f948002f310 comp rx=0 tx=0).stop 2024-04-25T13:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.052+0000 7f948d7e5640 1 -- 172.21.15.31:0/598762518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9488107280 msgr2=0x7f948819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.052+0000 7f948d7e5640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9488107280 0x7f948819e930 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c970 tx=0x7f947c00ce30 comp rx=0 tx=0).stop 2024-04-25T13:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.053+0000 7f948d7e5640 1 -- 172.21.15.31:0/598762518 shutdown_connections 2024-04-25T13:29:39.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.053+0000 7f948d7e5640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f94700a3d20 0x7f94700a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.053+0000 7f948d7e5640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881985a0 0x7f9488198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.053+0000 7f948d7e5640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9488107280 0x7f948819e930 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.053+0000 7f948d7e5640 1 --2- 172.21.15.31:0/598762518 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9488102240 0x7f948819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.053+0000 7f948d7e5640 1 -- 172.21.15.31:0/598762518 >> 172.21.15.31:0/598762518 conn(0x7f94880790e0 msgr2=0x7f94881003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:39.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.053+0000 7f948d7e5640 1 -- 172.21.15.31:0/598762518 shutdown_connections 2024-04-25T13:29:39.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.053+0000 7f948d7e5640 1 -- 172.21.15.31:0/598762518 wait complete. 2024-04-25T13:29:39.065 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:13:01.793580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:17:37.994125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:17:39.116456+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:29:39.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:29:39.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.170+0000 7fc332576640 1 -- 172.21.15.31:0/3042153474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc32c10e960 msgr2=0x7fc32c110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.170+0000 7fc332576640 1 --2- 172.21.15.31:0/3042153474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc32c10e960 0x7fc32c110d50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc324009ca0 tx=0x7fc32402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.170+0000 7fc332576640 1 -- 172.21.15.31:0/3042153474 shutdown_connections 2024-04-25T13:29:39.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.170+0000 7fc332576640 1 --2- 172.21.15.31:0/3042153474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc32c10e960 0x7fc32c110d50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.170+0000 7fc332576640 1 --2- 172.21.15.31:0/3042153474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c06aef0 0x7fc32c10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.170+0000 7fc332576640 1 --2- 172.21.15.31:0/3042153474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c06a5d0 0x7fc32c06a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.170+0000 7fc332576640 1 -- 172.21.15.31:0/3042153474 >> 172.21.15.31:0/3042153474 conn(0x7fc32c077eb0 msgr2=0x7fc32c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:39.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.170+0000 7fc332576640 1 -- 172.21.15.31:0/3042153474 shutdown_connections 2024-04-25T13:29:39.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.170+0000 7fc332576640 1 -- 172.21.15.31:0/3042153474 wait complete. 2024-04-25T13:29:39.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc332576640 1 Processor -- start 2024-04-25T13:29:39.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc332576640 1 -- start start 2024-04-25T13:29:39.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc332576640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c06a5d0 0x7fc32c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc332576640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc32c06aef0 0x7fc32c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc332576640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c10e960 0x7fc32c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc332576640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c114390 con 0x7fc32c06aef0 2024-04-25T13:29:39.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc332576640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c114210 con 0x7fc32c06a5d0 2024-04-25T13:29:39.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc332576640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c114510 con 0x7fc32c10e960 2024-04-25T13:29:39.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc32b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc32c06aef0 0x7fc32c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc32b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc32c06aef0 0x7fc32c1a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41628/0 (socket says 172.21.15.31:41628) 2024-04-25T13:29:39.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc32b7fe640 1 -- 172.21.15.31:0/3862946163 learned_addr learned my addr 172.21.15.31:0/3862946163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:29:39.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc32bfff640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c06a5d0 0x7fc32c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc330b13640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c10e960 0x7fc32c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc32b7fe640 1 -- 172.21.15.31:0/3862946163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c10e960 msgr2=0x7fc32c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc32b7fe640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c10e960 0x7fc32c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.171+0000 7fc32b7fe640 1 -- 172.21.15.31:0/3862946163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c06a5d0 msgr2=0x7fc32c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.172+0000 7fc32b7fe640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c06a5d0 0x7fc32c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.172+0000 7fc32b7fe640 1 -- 172.21.15.31:0/3862946163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c19afd0 con 0x7fc32c06aef0 2024-04-25T13:29:39.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.172+0000 7fc32b7fe640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc32c06aef0 0x7fc32c1a0b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc32000d950 tx=0x7fc32000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:39.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.172+0000 7fc3297fa640 1 -- 172.21.15.31:0/3862946163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320017070 con 0x7fc32c06aef0 2024-04-25T13:29:39.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.172+0000 7fc3297fa640 1 -- 172.21.15.31:0/3862946163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc320009c30 con 0x7fc32c06aef0 2024-04-25T13:29:39.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.172+0000 7fc332576640 1 -- 172.21.15.31:0/3862946163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c19b2c0 con 0x7fc32c06aef0 2024-04-25T13:29:39.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.172+0000 7fc3297fa640 1 -- 172.21.15.31:0/3862946163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32000f620 con 0x7fc32c06aef0 2024-04-25T13:29:39.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.172+0000 7fc332576640 1 -- 172.21.15.31:0/3862946163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c1a75d0 con 0x7fc32c06aef0 2024-04-25T13:29:39.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.173+0000 7fc332576640 1 -- 172.21.15.31:0/3862946163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32c101460 con 0x7fc32c06aef0 2024-04-25T13:29:39.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.174+0000 7fc3297fa640 1 -- 172.21.15.31:0/3862946163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc320010460 con 0x7fc32c06aef0 2024-04-25T13:29:39.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.175+0000 7fc3297fa640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fc3140a3d20 0x7fc3140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.175+0000 7fc32bfff640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fc3140a3d20 0x7fc3140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.175+0000 7fc3297fa640 1 -- 172.21.15.31:0/3862946163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fc3200d04d0 con 0x7fc32c06aef0 2024-04-25T13:29:39.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.175+0000 7fc32bfff640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fc3140a3d20 0x7fc3140a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc31c005fd0 tx=0x7fc31c005850 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:39.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.180+0000 7fc3297fa640 1 -- 172.21.15.31:0/3862946163 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc32009cb00 con 0x7fc32c06aef0 2024-04-25T13:29:39.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.365+0000 7fc332576640 1 -- 172.21.15.31:0/3862946163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc32c109af0 con 0x7fc32c06aef0 2024-04-25T13:29:39.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.366+0000 7fc3297fa640 1 -- 172.21.15.31:0/3862946163 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7fc320009de0 con 0x7fc32c06aef0 2024-04-25T13:29:39.367 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T13:29:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 -- 172.21.15.31:0/3862946163 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fc3140a3d20 msgr2=0x7fc3140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fc3140a3d20 0x7fc3140a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc31c005fd0 tx=0x7fc31c005850 comp rx=0 tx=0).stop 2024-04-25T13:29:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 -- 172.21.15.31:0/3862946163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc32c06aef0 msgr2=0x7fc32c1a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc32c06aef0 0x7fc32c1a0b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc32000d950 tx=0x7fc32000de10 comp rx=0 tx=0).stop 2024-04-25T13:29:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 -- 172.21.15.31:0/3862946163 shutdown_connections 2024-04-25T13:29:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fc3140a3d20 0x7fc3140a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c10e960 0x7fc32c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc32c06aef0 0x7fc32c1a0b90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 --2- 172.21.15.31:0/3862946163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc32c06a5d0 0x7fc32c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 -- 172.21.15.31:0/3862946163 >> 172.21.15.31:0/3862946163 conn(0x7fc32c077eb0 msgr2=0x7fc32c10fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 -- 172.21.15.31:0/3862946163 shutdown_connections 2024-04-25T13:29:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.368+0000 7fc332576640 1 -- 172.21.15.31:0/3862946163 wait complete. 2024-04-25T13:29:39.378 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:39 smithi031 ceph-mon[29457]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:39.379 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:39 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/598762518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:29:39.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:29:39.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:39 smithi066 ceph-mon[33137]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:39.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:39 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/598762518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:29:39.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:39 smithi136 ceph-mon[33038]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:39.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:39 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/598762518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:29:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.483+0000 7f75b47d8640 1 -- 172.21.15.31:0/830453974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75ac102b50 msgr2=0x7f75ac106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.483+0000 7f75b47d8640 1 --2- 172.21.15.31:0/830453974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75ac102b50 0x7f75ac106b90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f759c009900 tx=0x7f759c02a410 comp rx=0 tx=0).stop 2024-04-25T13:29:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 -- 172.21.15.31:0/830453974 shutdown_connections 2024-04-25T13:29:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 --2- 172.21.15.31:0/830453974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac107270 0x7f75ac10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 --2- 172.21.15.31:0/830453974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75ac102b50 0x7f75ac106b90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 --2- 172.21.15.31:0/830453974 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75ac1021a0 0x7f75ac102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 -- 172.21.15.31:0/830453974 >> 172.21.15.31:0/830453974 conn(0x7f75ac0fda40 msgr2=0x7f75ac0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 -- 172.21.15.31:0/830453974 shutdown_connections 2024-04-25T13:29:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 -- 172.21.15.31:0/830453974 wait complete. 2024-04-25T13:29:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 Processor -- start 2024-04-25T13:29:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 -- start start 2024-04-25T13:29:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac1021a0 0x7f75ac1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75ac107270 0x7f75ac1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75ac19a790 0x7f75ac19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac112190 con 0x7f75ac107270 2024-04-25T13:29:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac112010 con 0x7f75ac19a790 2024-04-25T13:29:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.484+0000 7f75b47d8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac112310 con 0x7f75ac1021a0 2024-04-25T13:29:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b1d73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75ac107270 0x7f75ac1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b1d73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75ac107270 0x7f75ac1a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41652/0 (socket says 172.21.15.31:41652) 2024-04-25T13:29:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b1d73640 1 -- 172.21.15.31:0/2273720927 learned_addr learned my addr 172.21.15.31:0/2273720927 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:29:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b2574640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac1021a0 0x7f75ac1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b2d75640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75ac19a790 0x7f75ac19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b1d73640 1 -- 172.21.15.31:0/2273720927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac1021a0 msgr2=0x7f75ac1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b1d73640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac1021a0 0x7f75ac1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b1d73640 1 -- 172.21.15.31:0/2273720927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75ac19a790 msgr2=0x7f75ac19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b1d73640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75ac19a790 0x7f75ac19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b1d73640 1 -- 172.21.15.31:0/2273720927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac19b370 con 0x7f75ac107270 2024-04-25T13:29:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b2574640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac1021a0 0x7f75ac1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:29:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b2d75640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75ac19a790 0x7f75ac19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:29:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b1d73640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75ac107270 0x7f75ac1a0b80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f759c032fd0 tx=0x7f759c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:39.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75a37fe640 1 -- 172.21.15.31:0/2273720927 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c03f030 con 0x7f75ac107270 2024-04-25T13:29:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75a37fe640 1 -- 172.21.15.31:0/2273720927 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f759c004490 con 0x7f75ac107270 2024-04-25T13:29:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75a37fe640 1 -- 172.21.15.31:0/2273720927 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c035070 con 0x7f75ac107270 2024-04-25T13:29:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b47d8640 1 -- 172.21.15.31:0/2273720927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ac1a7450 con 0x7f75ac107270 2024-04-25T13:29:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.485+0000 7f75b47d8640 1 -- 172.21.15.31:0/2273720927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ac1a7800 con 0x7f75ac107270 2024-04-25T13:29:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.486+0000 7f75b47d8640 1 -- 172.21.15.31:0/2273720927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ac0691c0 con 0x7f75ac107270 2024-04-25T13:29:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.488+0000 7f75a37fe640 1 -- 172.21.15.31:0/2273720927 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f759c042030 con 0x7f75ac107270 2024-04-25T13:29:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.488+0000 7f75a37fe640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f75980a3d20 0x7f75980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.488+0000 7f75b2574640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f75980a3d20 0x7f75980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.488+0000 7f75a37fe640 1 -- 172.21.15.31:0/2273720927 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f759c0f11b0 con 0x7f75ac107270 2024-04-25T13:29:39.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.489+0000 7f75b2574640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f75980a3d20 0x7f75980a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f75a800abe0 tx=0x7f75a800a5c0 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:39.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.492+0000 7f75a37fe640 1 -- 172.21.15.31:0/2273720927 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f759c0bd7e0 con 0x7f75ac107270 2024-04-25T13:29:39.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.677+0000 7f75b47d8640 1 -- 172.21.15.31:0/2273720927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f75ac102b50 con 0x7f75ac107270 2024-04-25T13:29:39.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.677+0000 7f75a37fe640 1 -- 172.21.15.31:0/2273720927 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) ==== 104+0+11 (secure 0 0 0) 0x7f759c0bd0e0 con 0x7f75ac107270 2024-04-25T13:29:39.679 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T13:29:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 -- 172.21.15.31:0/2273720927 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f75980a3d20 msgr2=0x7f75980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f75980a3d20 0x7f75980a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f75a800abe0 tx=0x7f75a800a5c0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 -- 172.21.15.31:0/2273720927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75ac107270 msgr2=0x7f75ac1a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75ac107270 0x7f75ac1a0b80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f759c032fd0 tx=0x7f759c004270 comp rx=0 tx=0).stop 2024-04-25T13:29:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 -- 172.21.15.31:0/2273720927 shutdown_connections 2024-04-25T13:29:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f75980a3d20 0x7f75980a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75ac19a790 0x7f75ac19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75ac107270 0x7f75ac1a0b80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 --2- 172.21.15.31:0/2273720927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac1021a0 0x7f75ac1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 -- 172.21.15.31:0/2273720927 >> 172.21.15.31:0/2273720927 conn(0x7f75ac0fda40 msgr2=0x7f75ac0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 -- 172.21.15.31:0/2273720927 shutdown_connections 2024-04-25T13:29:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.679+0000 7f75b47d8640 1 -- 172.21.15.31:0/2273720927 wait complete. 2024-04-25T13:29:39.691 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:29:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.795+0000 7fe87eec2640 1 -- 172.21.15.31:0/3901413114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe878107270 msgr2=0x7fe87810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.795+0000 7fe87eec2640 1 --2- 172.21.15.31:0/3901413114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe878107270 0x7fe87810ee40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe870009ca0 tx=0x7fe87002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.795+0000 7fe87eec2640 1 -- 172.21.15.31:0/3901413114 shutdown_connections 2024-04-25T13:29:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.795+0000 7fe87eec2640 1 --2- 172.21.15.31:0/3901413114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe878107270 0x7fe87810ee40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.795+0000 7fe87eec2640 1 --2- 172.21.15.31:0/3901413114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87806b1f0 0x7fe878106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.795+0000 7fe87eec2640 1 --2- 172.21.15.31:0/3901413114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe87806a840 0x7fe87806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.795+0000 7fe87eec2640 1 -- 172.21.15.31:0/3901413114 >> 172.21.15.31:0/3901413114 conn(0x7fe878077eb0 msgr2=0x7fe8780782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.795+0000 7fe87eec2640 1 -- 172.21.15.31:0/3901413114 shutdown_connections 2024-04-25T13:29:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.795+0000 7fe87eec2640 1 -- 172.21.15.31:0/3901413114 wait complete. 2024-04-25T13:29:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.795+0000 7fe87eec2640 1 Processor -- start 2024-04-25T13:29:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.795+0000 7fe87eec2640 1 -- start start 2024-04-25T13:29:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87eec2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe87806a840 0x7fe8781986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87eec2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe878107270 0x7fe878198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87eec2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe87819cf40 0x7fe87819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87eec2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781121b0 con 0x7fe87819cf40 2024-04-25T13:29:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87eec2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe878112030 con 0x7fe87806a840 2024-04-25T13:29:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87eec2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe878112330 con 0x7fe878107270 2024-04-25T13:29:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87d45f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe87819cf40 0x7fe87819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87d45f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe87819cf40 0x7fe87819d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41674/0 (socket says 172.21.15.31:41674) 2024-04-25T13:29:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87d45f640 1 -- 172.21.15.31:0/4241399010 learned_addr learned my addr 172.21.15.31:0/4241399010 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:29:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe877fff640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe878107270 0x7fe878198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87cc5e640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe87806a840 0x7fe8781986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87d45f640 1 -- 172.21.15.31:0/4241399010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe878107270 msgr2=0x7fe878198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87d45f640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe878107270 0x7fe878198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87d45f640 1 -- 172.21.15.31:0/4241399010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe87806a840 msgr2=0x7fe8781986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87d45f640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe87806a840 0x7fe8781986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87d45f640 1 -- 172.21.15.31:0/4241399010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87819da30 con 0x7fe87819cf40 2024-04-25T13:29:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe87d45f640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe87819cf40 0x7fe87819d3f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe870005cc0 tx=0x7fe87002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.796+0000 7fe875ffb640 1 -- 172.21.15.31:0/4241399010 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870003bf0 con 0x7fe87819cf40 2024-04-25T13:29:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.797+0000 7fe875ffb640 1 -- 172.21.15.31:0/4241399010 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe870003d90 con 0x7fe87819cf40 2024-04-25T13:29:39.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.797+0000 7fe87eec2640 1 -- 172.21.15.31:0/4241399010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87819dcc0 con 0x7fe87819cf40 2024-04-25T13:29:39.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.797+0000 7fe875ffb640 1 -- 172.21.15.31:0/4241399010 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87002fbc0 con 0x7fe87819cf40 2024-04-25T13:29:39.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.797+0000 7fe87eec2640 1 -- 172.21.15.31:0/4241399010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8781a54e0 con 0x7fe87819cf40 2024-04-25T13:29:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.798+0000 7fe87eec2640 1 -- 172.21.15.31:0/4241399010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8780691c0 con 0x7fe87819cf40 2024-04-25T13:29:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.800+0000 7fe875ffb640 1 -- 172.21.15.31:0/4241399010 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe87002fd60 con 0x7fe87819cf40 2024-04-25T13:29:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.800+0000 7fe875ffb640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8600a3d20 0x7fe8600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.800+0000 7fe87cc5e640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8600a3d20 0x7fe8600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.800+0000 7fe875ffb640 1 -- 172.21.15.31:0/4241399010 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fe8700f2110 con 0x7fe87819cf40 2024-04-25T13:29:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.801+0000 7fe87cc5e640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8600a3d20 0x7fe8600a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe868005fd0 tx=0x7fe868005850 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:39.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.804+0000 7fe875ffb640 1 -- 172.21.15.31:0/4241399010 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8700be850 con 0x7fe87819cf40 2024-04-25T13:29:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.994+0000 7fe87eec2640 1 -- 172.21.15.31:0/4241399010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe87806b1f0 con 0x7fe87819cf40 2024-04-25T13:29:39.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.994+0000 7fe875ffb640 1 -- 172.21.15.31:0/4241399010 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) ==== 100+0+11 (secure 0 0 0) 0x7fe8700be150 con 0x7fe87819cf40 2024-04-25T13:29:39.996 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T13:29:39.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.996+0000 7fe87eec2640 1 -- 172.21.15.31:0/4241399010 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8600a3d20 msgr2=0x7fe8600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.996+0000 7fe87eec2640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8600a3d20 0x7fe8600a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe868005fd0 tx=0x7fe868005850 comp rx=0 tx=0).stop 2024-04-25T13:29:39.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.996+0000 7fe87eec2640 1 -- 172.21.15.31:0/4241399010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe87819cf40 msgr2=0x7fe87819d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:39.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.996+0000 7fe87eec2640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe87819cf40 0x7fe87819d3f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe870005cc0 tx=0x7fe87002f380 comp rx=0 tx=0).stop 2024-04-25T13:29:39.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.997+0000 7fe87eec2640 1 -- 172.21.15.31:0/4241399010 shutdown_connections 2024-04-25T13:29:39.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.997+0000 7fe87eec2640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe8600a3d20 0x7fe8600a61e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.997+0000 7fe87eec2640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe87819cf40 0x7fe87819d3f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.997+0000 7fe87eec2640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe878107270 0x7fe878198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.997+0000 7fe87eec2640 1 --2- 172.21.15.31:0/4241399010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe87806a840 0x7fe8781986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.997+0000 7fe87eec2640 1 -- 172.21.15.31:0/4241399010 >> 172.21.15.31:0/4241399010 conn(0x7fe878077eb0 msgr2=0x7fe8780fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.997+0000 7fe87eec2640 1 -- 172.21.15.31:0/4241399010 shutdown_connections 2024-04-25T13:29:39.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:39.997+0000 7fe87eec2640 1 -- 172.21.15.31:0/4241399010 wait complete. 2024-04-25T13:29:40.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T13:29:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.109+0000 7fd8498a7640 1 -- 172.21.15.31:0/2841764514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd84410c570 msgr2=0x7fd84410e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.109+0000 7fd8498a7640 1 --2- 172.21.15.31:0/2841764514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd84410c570 0x7fd84410e9e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009ca0 tx=0x7fd83c02a950 comp rx=0 tx=0).stop 2024-04-25T13:29:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.109+0000 7fd8498a7640 1 -- 172.21.15.31:0/2841764514 shutdown_connections 2024-04-25T13:29:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.109+0000 7fd8498a7640 1 --2- 172.21.15.31:0/2841764514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd84410c570 0x7fd84410e9e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:40.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.109+0000 7fd8498a7640 1 --2- 172.21.15.31:0/2841764514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd844103b50 0x7fd84410c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:40.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.109+0000 7fd8498a7640 1 --2- 172.21.15.31:0/2841764514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd844103230 0x7fd844103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:40.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.109+0000 7fd8498a7640 1 -- 172.21.15.31:0/2841764514 >> 172.21.15.31:0/2841764514 conn(0x7fd844077fb0 msgr2=0x7fd8440783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:40.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.109+0000 7fd8498a7640 1 -- 172.21.15.31:0/2841764514 shutdown_connections 2024-04-25T13:29:40.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.109+0000 7fd8498a7640 1 -- 172.21.15.31:0/2841764514 wait complete. 2024-04-25T13:29:40.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8498a7640 1 Processor -- start 2024-04-25T13:29:40.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8498a7640 1 -- start start 2024-04-25T13:29:40.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8498a7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd844103230 0x7fd8441986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:40.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8498a7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd844103b50 0x7fd844198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:40.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8498a7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84410c570 0x7fd84419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:40.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8498a7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441121b0 con 0x7fd844103b50 2024-04-25T13:29:40.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8498a7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd844112030 con 0x7fd84410c570 2024-04-25T13:29:40.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8498a7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd844112330 con 0x7fd844103230 2024-04-25T13:29:40.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8427fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd844103b50 0x7fd844198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:40.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8427fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd844103b50 0x7fd844198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41694/0 (socket says 172.21.15.31:41694) 2024-04-25T13:29:40.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8427fc640 1 -- 172.21.15.31:0/3633910355 learned_addr learned my addr 172.21.15.31:0/3633910355 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:29:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd8437fe640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84410c570 0x7fd84419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.110+0000 7fd842ffd640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd844103230 0x7fd8441986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.111+0000 7fd8427fc640 1 -- 172.21.15.31:0/3633910355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd844103230 msgr2=0x7fd8441986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:40.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.111+0000 7fd8427fc640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd844103230 0x7fd8441986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.111+0000 7fd8427fc640 1 -- 172.21.15.31:0/3633910355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84410c570 msgr2=0x7fd84419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.111+0000 7fd8427fc640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84410c570 0x7fd84419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.111+0000 7fd8427fc640 1 -- 172.21.15.31:0/3633910355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84419d690 con 0x7fd844103b50 2024-04-25T13:29:40.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.111+0000 7fd8427fc640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd844103b50 0x7fd844198c30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c890 tx=0x7fd83800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:40.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.111+0000 7fd8488a5640 1 -- 172.21.15.31:0/3633910355 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838004220 con 0x7fd844103b50 2024-04-25T13:29:40.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.111+0000 7fd8498a7640 1 -- 172.21.15.31:0/3633910355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84419d980 con 0x7fd844103b50 2024-04-25T13:29:40.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.111+0000 7fd8488a5640 1 -- 172.21.15.31:0/3633910355 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8380043c0 con 0x7fd844103b50 2024-04-25T13:29:40.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.111+0000 7fd8488a5640 1 -- 172.21.15.31:0/3633910355 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838004a00 con 0x7fd844103b50 2024-04-25T13:29:40.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.111+0000 7fd8498a7640 1 -- 172.21.15.31:0/3633910355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8441a5210 con 0x7fd844103b50 2024-04-25T13:29:40.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.112+0000 7fd8498a7640 1 -- 172.21.15.31:0/3633910355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8440691c0 con 0x7fd844103b50 2024-04-25T13:29:40.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.113+0000 7fd8488a5640 1 -- 172.21.15.31:0/3633910355 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd8380079b0 con 0x7fd844103b50 2024-04-25T13:29:40.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.114+0000 7fd8488a5640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd82c0a3d20 0x7fd82c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:29:40.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.114+0000 7fd842ffd640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd82c0a3d20 0x7fd82c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:29:40.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.114+0000 7fd8488a5640 1 -- 172.21.15.31:0/3633910355 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fd8380d04a0 con 0x7fd844103b50 2024-04-25T13:29:40.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.115+0000 7fd842ffd640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd82c0a3d20 0x7fd82c0a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd83400bbe0 tx=0x7fd83400b390 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:29:40.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.119+0000 7fd8488a5640 1 -- 172.21.15.31:0/3633910355 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd838014030 con 0x7fd844103b50 2024-04-25T13:29:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.357+0000 7fd8498a7640 1 -- 172.21.15.31:0/3633910355 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd844107160 con 0x7fd844103b50 2024-04-25T13:29:40.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.366+0000 7fd8488a5640 1 -- 172.21.15.31:0/3633910355 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746060 (secure 0 0 0) 0x7fd83809c480 con 0x7fd844103b50 2024-04-25T13:29:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:29:40.377 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T13:29:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 26, 2024-04-25T13:29:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 24922, 2024-04-25T13:29:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "x", 2024-04-25T13:29:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2024-04-25T13:29:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T13:29:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T13:29:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-04-25T13:29:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 332946768 2024-04-25T13:29:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-04-25T13:29:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-04-25T13:29:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 332946768 2024-04-25T13:29:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:29:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.31:6801/332946768", 2024-04-25T13:29:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2024-04-25T13:19:35.271347+0000", 2024-04-25T13:29:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T13:29:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2024-04-25T13:29:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2024-04-25T13:29:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24887, 2024-04-25T13:29:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "y", 2024-04-25T13:29:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540701547738038271, 2024-04-25T13:29:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-04-25T13:29:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-04-25T13:29:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:29:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-04-25T13:29:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-04-25T13:29:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:29:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-04-25T13:29:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-04-25T13:29:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-04-25T13:29:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-04-25T13:29:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-04-25T13:29:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-04-25T13:29:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-04-25T13:29:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-04-25T13:29:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-04-25T13:29:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-04-25T13:29:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-04-25T13:29:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-04-25T13:29:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-04-25T13:29:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-04-25T13:29:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-04-25T13:29:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-04-25T13:29:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:29:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-04-25T13:29:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-04-25T13:29:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:29:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-04-25T13:29:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-04-25T13:29:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-04-25T13:29:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-04-25T13:29:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-04-25T13:29:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-04-25T13:29:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:29:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-04-25T13:29:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-04-25T13:29:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-04-25T13:29:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:29:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:29:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-04-25T13:29:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-04-25T13:29:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T13:29:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T13:29:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:29:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:29:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-04-25T13:29:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:29:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-04-25T13:29:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-04-25T13:29:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:29:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-04-25T13:29:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-04-25T13:29:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:29:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:29:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:29:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-04-25T13:29:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-04-25T13:29:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-04-25T13:29:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-04-25T13:29:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-04-25T13:29:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:29:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:29:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-04-25T13:29:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-04-25T13:29:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-04-25T13:29:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:29:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:29:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-04-25T13:29:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-04-25T13:29:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T13:29:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T13:29:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:29:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:29:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-04-25T13:29:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-04-25T13:29:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:29:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T13:29:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T13:29:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-04-25T13:29:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-04-25T13:29:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-04-25T13:29:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-04-25T13:29:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-04-25T13:29:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-04-25T13:29:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-04-25T13:29:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-04-25T13:29:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-04-25T13:29:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:29:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:29:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-04-25T13:29:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-04-25T13:29:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:29:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:29:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-04-25T13:29:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:29:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:29:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-04-25T13:29:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-04-25T13:29:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-04-25T13:29:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T13:29:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:29:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-04-25T13:29:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-04-25T13:29:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-04-25T13:29:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:29:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-04-25T13:29:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-04-25T13:29:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-04-25T13:29:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:29:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-04-25T13:29:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-04-25T13:29:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:29:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:29:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-04-25T13:29:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-04-25T13:29:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:29:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:29:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:29:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-04-25T13:29:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:29:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-04-25T13:29:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-04-25T13:29:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-04-25T13:29:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-04-25T13:29:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-04-25T13:29:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:29:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-04-25T13:29:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-04-25T13:29:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:29:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-04-25T13:29:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-04-25T13:29:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:29:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-04-25T13:29:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-04-25T13:29:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:29:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-04-25T13:29:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-04-25T13:29:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:29:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-04-25T13:29:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-04-25T13:29:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:29:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:29:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-04-25T13:29:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:29:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:29:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-04-25T13:29:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-04-25T13:29:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-04-25T13:29:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:29:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-04-25T13:29:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-04-25T13:29:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:29:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-04-25T13:29:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-04-25T13:29:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:29:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:29:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:29:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-04-25T13:29:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:29:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:29:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:29:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-04-25T13:29:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:29:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:29:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-04-25T13:29:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-04-25T13:29:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-04-25T13:29:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:29:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-04-25T13:29:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-04-25T13:29:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-04-25T13:29:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:29:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-04-25T13:29:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-04-25T13:29:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-04-25T13:29:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:29:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-04-25T13:29:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-04-25T13:29:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:29:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-04-25T13:29:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-04-25T13:29:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-04-25T13:29:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:29:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-04-25T13:29:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-04-25T13:29:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:29:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-04-25T13:29:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-04-25T13:29:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-04-25T13:29:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:29:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:29:40.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.482 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-04-25T13:29:40.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:29:40.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:29:40.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.483 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:29:40.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.484 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-04-25T13:29:40.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-04-25T13:29:40.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:29:40.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.486 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-04-25T13:29:40.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:29:40.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:40.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:29:40.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.487 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:29:40.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:29:40.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.488 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-04-25T13:29:40.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:29:40.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-04-25T13:29:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-04-25T13:29:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-04-25T13:29:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:29:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-04-25T13:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-04-25T13:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-04-25T13:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-04-25T13:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-04-25T13:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-04-25T13:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-04-25T13:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-04-25T13:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-04-25T13:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-04-25T13:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-04-25T13:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-04-25T13:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-04-25T13:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-04-25T13:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-04-25T13:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-04-25T13:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-04-25T13:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-04-25T13:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-04-25T13:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-04-25T13:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-04-25T13:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-04-25T13:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-04-25T13:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-04-25T13:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T13:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T13:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-04-25T13:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-04-25T13:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-04-25T13:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-04-25T13:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-04-25T13:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-04-25T13:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-04-25T13:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-04-25T13:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-04-25T13:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-04-25T13:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-04-25T13:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-04-25T13:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-04-25T13:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-04-25T13:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-04-25T13:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-04-25T13:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-04-25T13:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-04-25T13:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-04-25T13:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-04-25T13:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-04-25T13:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-04-25T13:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-04-25T13:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-04-25T13:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-04-25T13:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-04-25T13:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-04-25T13:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-04-25T13:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-04-25T13:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-04-25T13:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-04-25T13:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-04-25T13:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-04-25T13:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-04-25T13:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-04-25T13:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-04-25T13:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T13:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-04-25T13:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T13:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "[]", 2024-04-25T13:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "{}", 2024-04-25T13:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-04-25T13:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-04-25T13:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T13:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-04-25T13:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-04-25T13:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-04-25T13:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-04-25T13:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-04-25T13:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-04-25T13:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-04-25T13:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-04-25T13:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-04-25T13:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T13:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T13:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-04-25T13:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-04-25T13:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-04-25T13:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T13:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T13:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T13:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T13:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T13:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-04-25T13:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T13:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T13:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-04-25T13:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-04-25T13:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-04-25T13:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T13:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T13:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-04-25T13:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-04-25T13:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T13:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T13:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T13:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T13:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T13:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-04-25T13:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-04-25T13:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T13:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-04-25T13:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-04-25T13:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-04-25T13:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-04-25T13:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-04-25T13:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-04-25T13:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-04-25T13:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-04-25T13:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-04-25T13:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-04-25T13:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-04-25T13:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-04-25T13:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-04-25T13:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-04-25T13:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T13:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-04-25T13:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-04-25T13:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-04-25T13:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-04-25T13:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-04-25T13:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-04-25T13:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-04-25T13:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-04-25T13:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T13:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-04-25T13:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-04-25T13:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-04-25T13:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-04-25T13:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-04-25T13:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-04-25T13:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-04-25T13:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-04-25T13:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-04-25T13:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-04-25T13:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-04-25T13:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-04-25T13:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-04-25T13:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-04-25T13:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-04-25T13:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-04-25T13:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-04-25T13:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-04-25T13:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-04-25T13:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-04-25T13:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-04-25T13:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T13:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T13:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-04-25T13:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-04-25T13:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-04-25T13:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-04-25T13:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-04-25T13:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-04-25T13:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-04-25T13:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-04-25T13:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-04-25T13:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-04-25T13:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-04-25T13:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-04-25T13:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-04-25T13:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-04-25T13:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-04-25T13:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-04-25T13:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-04-25T13:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-04-25T13:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-04-25T13:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-04-25T13:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-04-25T13:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-04-25T13:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-04-25T13:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-04-25T13:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-04-25T13:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-04-25T13:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-04-25T13:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-04-25T13:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-04-25T13:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-04-25T13:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-04-25T13:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-04-25T13:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-04-25T13:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-04-25T13:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-04-25T13:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-04-25T13:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-04-25T13:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T13:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-04-25T13:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-04-25T13:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-04-25T13:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T13:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T13:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-04-25T13:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-04-25T13:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-04-25T13:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-04-25T13:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-04-25T13:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-04-25T13:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-04-25T13:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-04-25T13:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T13:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-04-25T13:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-04-25T13:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-04-25T13:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-04-25T13:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T13:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T13:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-04-25T13:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-04-25T13:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-04-25T13:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-04-25T13:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-04-25T13:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-04-25T13:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-04-25T13:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-04-25T13:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-04-25T13:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-04-25T13:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-04-25T13:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-04-25T13:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-04-25T13:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-04-25T13:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T13:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T13:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T13:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T13:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T13:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-04-25T13:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-04-25T13:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-04-25T13:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-04-25T13:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T13:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T13:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-04-25T13:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-04-25T13:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-04-25T13:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T13:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T13:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T13:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T13:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T13:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T13:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-04-25T13:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-04-25T13:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-04-25T13:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-04-25T13:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-04-25T13:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-04-25T13:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T13:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T13:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T13:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T13:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T13:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T13:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-04-25T13:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-04-25T13:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-04-25T13:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-04-25T13:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-04-25T13:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-04-25T13:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-04-25T13:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-04-25T13:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-04-25T13:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-04-25T13:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-04-25T13:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-04-25T13:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-04-25T13:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-04-25T13:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-04-25T13:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-04-25T13:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-04-25T13:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-04-25T13:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-04-25T13:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-04-25T13:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-04-25T13:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-04-25T13:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-04-25T13:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-04-25T13:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-04-25T13:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-04-25T13:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-04-25T13:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-04-25T13:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-04-25T13:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-04-25T13:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-04-25T13:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-04-25T13:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-04-25T13:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-04-25T13:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-04-25T13:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-04-25T13:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-04-25T13:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-04-25T13:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-04-25T13:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-04-25T13:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-04-25T13:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-04-25T13:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-04-25T13:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-04-25T13:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-04-25T13:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-04-25T13:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-04-25T13:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-04-25T13:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-04-25T13:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-04-25T13:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-04-25T13:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-04-25T13:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-04-25T13:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-04-25T13:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-04-25T13:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-04-25T13:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-04-25T13:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-04-25T13:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-04-25T13:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-04-25T13:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T13:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T13:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-04-25T13:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-04-25T13:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-04-25T13:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-04-25T13:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-04-25T13:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-04-25T13:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-04-25T13:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-04-25T13:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-04-25T13:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-04-25T13:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-04-25T13:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-04-25T13:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-04-25T13:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-04-25T13:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-04-25T13:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-04-25T13:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T13:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-04-25T13:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-04-25T13:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-04-25T13:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-04-25T13:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-04-25T13:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-04-25T13:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-04-25T13:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-04-25T13:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-04-25T13:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-04-25T13:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-04-25T13:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-04-25T13:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-04-25T13:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-04-25T13:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-04-25T13:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-04-25T13:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-04-25T13:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-04-25T13:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-04-25T13:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-04-25T13:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24916, 2024-04-25T13:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "z", 2024-04-25T13:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540701547738038271, 2024-04-25T13:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-04-25T13:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-04-25T13:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-04-25T13:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-04-25T13:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-04-25T13:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-04-25T13:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-04-25T13:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-04-25T13:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-04-25T13:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-04-25T13:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-04-25T13:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-04-25T13:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-04-25T13:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-04-25T13:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-04-25T13:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-04-25T13:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-04-25T13:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-04-25T13:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:29:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-04-25T13:29:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-04-25T13:29:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:29:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-04-25T13:29:41.148 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-04-25T13:29:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-04-25T13:29:41.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-04-25T13:29:41.150 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-04-25T13:29:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-04-25T13:29:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:29:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-04-25T13:29:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-04-25T13:29:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-04-25T13:29:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:29:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:29:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.155 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-04-25T13:29:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-04-25T13:29:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T13:29:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T13:29:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:29:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:29:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-04-25T13:29:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:29:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-04-25T13:29:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-04-25T13:29:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:29:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-04-25T13:29:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-04-25T13:29:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:29:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:29:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:29:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-04-25T13:29:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-04-25T13:29:41.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-04-25T13:29:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-04-25T13:29:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-04-25T13:29:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:29:41.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:29:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-04-25T13:29:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-04-25T13:29:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-04-25T13:29:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:29:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:29:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.163 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-04-25T13:29:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-04-25T13:29:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T13:29:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T13:29:41.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:29:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:29:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.170 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-04-25T13:29:41.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-04-25T13:29:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:29:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T13:29:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T13:29:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-04-25T13:29:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-04-25T13:29:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-04-25T13:29:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-04-25T13:29:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-04-25T13:29:41.176 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-04-25T13:29:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-04-25T13:29:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-04-25T13:29:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-04-25T13:29:41.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:29:41.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:29:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.180 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-04-25T13:29:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-04-25T13:29:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:29:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:29:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-04-25T13:29:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:29:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:41.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:29:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.183 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-04-25T13:29:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.184 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-04-25T13:29:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-04-25T13:29:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T13:29:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:29:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-04-25T13:29:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-04-25T13:29:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-04-25T13:29:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:29:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-04-25T13:29:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-04-25T13:29:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-04-25T13:29:41.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:29:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-04-25T13:29:41.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-04-25T13:29:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:29:41.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:29:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-04-25T13:29:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-04-25T13:29:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:29:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.192 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:29:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:29:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-04-25T13:29:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:29:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-04-25T13:29:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-04-25T13:29:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-04-25T13:29:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-04-25T13:29:41.195 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-04-25T13:29:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:29:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-04-25T13:29:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-04-25T13:29:41.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:29:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-04-25T13:29:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-04-25T13:29:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:29:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.199 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-04-25T13:29:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-04-25T13:29:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:29:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-04-25T13:29:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-04-25T13:29:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:29:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-04-25T13:29:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-04-25T13:29:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:29:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:29:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-04-25T13:29:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:29:41.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:29:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-04-25T13:29:41.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-04-25T13:29:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-04-25T13:29:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:29:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-04-25T13:29:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-04-25T13:29:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:29:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-04-25T13:29:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-04-25T13:29:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:29:41.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:29:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:29:41.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-04-25T13:29:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:29:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:29:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:29:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-04-25T13:29:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:29:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:29:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-04-25T13:29:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-04-25T13:29:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-04-25T13:29:41.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:29:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-04-25T13:29:41.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-04-25T13:29:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-04-25T13:29:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:29:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-04-25T13:29:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-04-25T13:29:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-04-25T13:29:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:29:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.218 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-04-25T13:29:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-04-25T13:29:41.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:29:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-04-25T13:29:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-04-25T13:29:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-04-25T13:29:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:29:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.222 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-04-25T13:29:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-04-25T13:29:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:29:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-04-25T13:29:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-04-25T13:29:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-04-25T13:29:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:29:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:29:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-04-25T13:29:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:29:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:29:41.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:29:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-04-25T13:29:41.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-04-25T13:29:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:29:41.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-04-25T13:29:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:29:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:29:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:29:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:29:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-04-25T13:29:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:29:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-04-25T13:29:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-04-25T13:29:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-04-25T13:29:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:29:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-04-25T13:29:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-04-25T13:29:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-04-25T13:29:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:29:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-04-25T13:29:41.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-04-25T13:29:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:29:41.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-04-25T13:29:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-04-25T13:29:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:29:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-04-25T13:29:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:29:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:29:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:29:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-04-25T13:29:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-04-25T13:29:41.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:29:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-04-25T13:29:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-04-25T13:29:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:29:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.243 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-04-25T13:29:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-04-25T13:29:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:29:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.246 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-04-25T13:29:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-04-25T13:29:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:29:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:29:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.252 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:29:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:29:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:29:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-04-25T13:29:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.257 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:29:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-04-25T13:29:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-04-25T13:29:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:29:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-04-25T13:29:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-04-25T13:29:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:29:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-04-25T13:29:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-04-25T13:29:41.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-04-25T13:29:41.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T13:29:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T13:29:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-04-25T13:29:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-04-25T13:29:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-04-25T13:29:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:29:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-04-25T13:29:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-04-25T13:29:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-04-25T13:29:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:29:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-04-25T13:29:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:29:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:29:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:29:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-04-25T13:29:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-04-25T13:29:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:29:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-04-25T13:29:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-04-25T13:29:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:29:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-04-25T13:29:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-04-25T13:29:41.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:29:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-04-25T13:29:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-04-25T13:29:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.273 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:29:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.274 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-04-25T13:29:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:29:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:29:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-04-25T13:29:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-04-25T13:29:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-04-25T13:29:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:29:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-04-25T13:29:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-04-25T13:29:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:29:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-04-25T13:29:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-04-25T13:29:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:29:41.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.281 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-04-25T13:29:41.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-04-25T13:29:41.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:29:41.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.282 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-04-25T13:29:41.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:29:41.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:29:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-04-25T13:29:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:29:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:29:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-04-25T13:29:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:29:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:29:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-04-25T13:29:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.289 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.293 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.297 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-04-25T13:29:41.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-04-25T13:29:41.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-04-25T13:29:41.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:29:41.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.299 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-04-25T13:29:41.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-04-25T13:29:41.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.300 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-04-25T13:29:41.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:29:41.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-04-25T13:29:41.302 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.302 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.302 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.302 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:29:41.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:29:41.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:41.303 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.303 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.303 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.304 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:29:41.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:29:41.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.305 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.305 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.306 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:29:41.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:29:41.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.307 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.307 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:29:41.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:29:41.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.309 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:29:41.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:29:41.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.311 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:29:41.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:29:41.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.312 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:29:41.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:29:41.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.314 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:29:41.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:29:41.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.316 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:29:41.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:29:41.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.318 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:29:41.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:29:41.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.319 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:29:41.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:29:41.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.321 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:29:41.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:29:41.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.323 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:29:41.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:29:41.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.324 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:29:41.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:29:41.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.326 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:29:41.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:29:41.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T13:29:41.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.328 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:29:41.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:29:41.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.330 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-04-25T13:29:41.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:29:41.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.331 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:29:41.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:29:41.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T13:29:41.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.333 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:29:41.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:29:41.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.335 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:29:41.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:29:41.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.336 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:29:41.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:29:41.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.337 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.338 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:29:41.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:29:41.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.340 INFO:teuthology.orchestra.run.smithi031.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:29:41.340 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:29:41.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "[]", 2024-04-25T13:29:41.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.341 INFO:teuthology.orchestra.run.smithi031.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:29:41.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:29:41.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "{}", 2024-04-25T13:29:41.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.343 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.343 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:29:41.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:29:41.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.345 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:29:41.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:29:41.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.345 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.346 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:29:41.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:29:41.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.348 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:29:41.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:29:41.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.350 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:29:41.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:29:41.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.351 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:29:41.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:29:41.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.353 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:29:41.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:29:41.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.355 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:29:41.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:29:41.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.356 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:29:41.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:29:41.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.358 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:29:41.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:29:41.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.359 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:29:41.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:29:41.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:29:41.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.361 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:29:41.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:29:41.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:41.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.362 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.363 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:29:41.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:29:41.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-04-25T13:29:41.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.364 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:29:41.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:29:41.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-04-25T13:29:41.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.366 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:29:41.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:29:41.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.368 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:29:41.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:29:41.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T13:29:41.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.369 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:29:41.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:29:41.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.370 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.371 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:29:41.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:29:41.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.373 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:29:41.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:29:41.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.375 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:29:41.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:29:41.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.375 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:41.375 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.376 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:29:41.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:29:41.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:41.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.378 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:29:41.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:29:41.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:41.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-04-25T13:29:41.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-04-25T13:29:41.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.382 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-04-25T13:29:41.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-04-25T13:29:41.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.384 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-04-25T13:29:41.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-04-25T13:29:41.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:29:41.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.385 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-04-25T13:29:41.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-04-25T13:29:41.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-04-25T13:29:41.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.387 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T13:29:41.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T13:29:41.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.388 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.390 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.390 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.390 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.390 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.390 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.390 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.391 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.393 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.394 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.394 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.394 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.394 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.394 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.394 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.394 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.395 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.397 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-04-25T13:29:41.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-04-25T13:29:41.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.397 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-04-25T13:29:41.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.398 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:29:41.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:29:41.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.400 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T13:29:41.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T13:29:41.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T13:29:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.402 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T13:29:41.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T13:29:41.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-04-25T13:29:41.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.403 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.403 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T13:29:41.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T13:29:41.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.405 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-04-25T13:29:41.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-04-25T13:29:41.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-04-25T13:29:41.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.407 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T13:29:41.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T13:29:41.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-04-25T13:29:41.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.408 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.408 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-04-25T13:29:41.408 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.409 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T13:29:41.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T13:29:41.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T13:29:41.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T13:29:41.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T13:29:41.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.410 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-04-25T13:29:41.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-04-25T13:29:41.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T13:29:41.412 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.412 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.412 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-04-25T13:29:41.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-04-25T13:29:41.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-04-25T13:29:41.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.414 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-04-25T13:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-04-25T13:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-04-25T13:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-04-25T13:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-04-25T13:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-04-25T13:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-04-25T13:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-04-25T13:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-04-25T13:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-04-25T13:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-04-25T13:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T13:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-04-25T13:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-04-25T13:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-04-25T13:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-04-25T13:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-04-25T13:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-04-25T13:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-04-25T13:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-04-25T13:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T13:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-04-25T13:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-04-25T13:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-04-25T13:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-04-25T13:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-04-25T13:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-04-25T13:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-04-25T13:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-04-25T13:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-04-25T13:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-04-25T13:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-04-25T13:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-04-25T13:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-04-25T13:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-04-25T13:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-04-25T13:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-04-25T13:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-04-25T13:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-04-25T13:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-04-25T13:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-04-25T13:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-04-25T13:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T13:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T13:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-04-25T13:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-04-25T13:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-04-25T13:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-04-25T13:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-04-25T13:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-04-25T13:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-04-25T13:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-04-25T13:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-04-25T13:29:41.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-04-25T13:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-04-25T13:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-04-25T13:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-04-25T13:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-04-25T13:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-04-25T13:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-04-25T13:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-04-25T13:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-04-25T13:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-04-25T13:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-04-25T13:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-04-25T13:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-04-25T13:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-04-25T13:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-04-25T13:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-04-25T13:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-04-25T13:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-04-25T13:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-04-25T13:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-04-25T13:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-04-25T13:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-04-25T13:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-04-25T13:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-04-25T13:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-04-25T13:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-04-25T13:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-04-25T13:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.532 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.532 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.532 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.533 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.533 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.533 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.533 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.533 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.533 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.533 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.533 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.533 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.534 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.535 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.535 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.536 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.536 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.536 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.536 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.537 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.537 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.537 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.537 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.537 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.537 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.537 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.538 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.538 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.538 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.539 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.539 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.539 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.540 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-04-25T13:29:41.540 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.540 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.540 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.541 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.542 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.542 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.542 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.542 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.542 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.542 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.542 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.543 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.543 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.543 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.544 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.544 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.545 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.545 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.546 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.546 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.546 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.546 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.546 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.546 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.546 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.546 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.546 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.547 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.547 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.548 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.548 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.548 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.548 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.548 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T13:29:41.549 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.549 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.549 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.549 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-04-25T13:29:41.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-04-25T13:29:41.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.550 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.550 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:29:41.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.551 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.551 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.552 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.552 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.552 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.552 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.553 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.553 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.553 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.554 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.555 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.556 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.556 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.556 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.556 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.556 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.556 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.557 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.557 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.557 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.557 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.558 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.558 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.559 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.559 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.559 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.559 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.559 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-04-25T13:29:41.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T13:29:41.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.560 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.560 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.560 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.560 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T13:29:41.560 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-04-25T13:29:41.561 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-04-25T13:29:41.561 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-04-25T13:29:41.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-04-25T13:29:41.562 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.562 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.562 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.562 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.565 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.565 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.565 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-04-25T13:29:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-04-25T13:29:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-04-25T13:29:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-04-25T13:29:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T13:29:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-04-25T13:29:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-04-25T13:29:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:29:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-04-25T13:29:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-04-25T13:29:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:29:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:29:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T13:29:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T13:29:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-04-25T13:29:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-04-25T13:29:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-04-25T13:29:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:29:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:29:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-04-25T13:29:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-04-25T13:29:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-04-25T13:29:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-04-25T13:29:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-04-25T13:29:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:29:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:29:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-04-25T13:29:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-04-25T13:29:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:29:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:29:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-04-25T13:29:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-04-25T13:29:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-04-25T13:29:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-04-25T13:29:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T13:29:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T13:29:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T13:29:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:29:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T13:29:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T13:29:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-04-25T13:29:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:29:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-04-25T13:29:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-04-25T13:29:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-04-25T13:29:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T13:29:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T13:29:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-04-25T13:29:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-04-25T13:29:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-04-25T13:29:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T13:29:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T13:29:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T13:29:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T13:29:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T13:29:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T13:29:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-04-25T13:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:29:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:29:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-04-25T13:29:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-04-25T13:29:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-04-25T13:29:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-04-25T13:29:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-04-25T13:29:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T13:29:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T13:29:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T13:29:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T13:29:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T13:29:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T13:29:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-04-25T13:29:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-04-25T13:29:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-04-25T13:29:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-04-25T13:29:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-04-25T13:29:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:29:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-04-25T13:29:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-04-25T13:29:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-04-25T13:29:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-04-25T13:29:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-04-25T13:29:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-04-25T13:29:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-04-25T13:29:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-04-25T13:29:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.700 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-04-25T13:29:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-04-25T13:29:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-04-25T13:29:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-04-25T13:29:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-04-25T13:29:41.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-04-25T13:29:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-04-25T13:29:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-04-25T13:29:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-04-25T13:29:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-04-25T13:29:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.708 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-04-25T13:29:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-04-25T13:29:41.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-04-25T13:29:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-04-25T13:29:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-04-25T13:29:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.712 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-04-25T13:29:41.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-04-25T13:29:41.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.713 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.713 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.713 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.713 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-04-25T13:29:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-04-25T13:29:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-04-25T13:29:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.716 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-04-25T13:29:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-04-25T13:29:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.717 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:29:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-04-25T13:29:41.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-04-25T13:29:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:29:41.719 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.721 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.725 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-04-25T13:29:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.729 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.730 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.734 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-04-25T13:29:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.739 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.742 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.743 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-04-25T13:29:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-04-25T13:29:41.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-04-25T13:29:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:29:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-04-25T13:29:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.750 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.751 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.752 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.756 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-04-25T13:29:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.759 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-04-25T13:29:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-04-25T13:29:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:29:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-04-25T13:29:41.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-04-25T13:29:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:29:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-04-25T13:29:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-04-25T13:29:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.763 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:29:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-04-25T13:29:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-04-25T13:29:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-04-25T13:29:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-04-25T13:29:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:29:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.769 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-04-25T13:29:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-04-25T13:29:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-04-25T13:29:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-04-25T13:29:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.772 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-04-25T13:29:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-04-25T13:29:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:29:41.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T13:29:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T13:29:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.776 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-04-25T13:29:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-04-25T13:29:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-04-25T13:29:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-04-25T13:29:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-04-25T13:29:41.778 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-04-25T13:29:41.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-04-25T13:29:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.780 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-04-25T13:29:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-04-25T13:29:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.782 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.784 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.790 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-04-25T13:29:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-04-25T13:29:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.791 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-04-25T13:29:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-04-25T13:29:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-04-25T13:29:41.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-04-25T13:29:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:29:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.795 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-04-25T13:29:41.796 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.797 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.798 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.800 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.802 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.803 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T13:29:41.806 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.806 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.806 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.806 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.806 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.806 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.807 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.807 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.807 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.807 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.807 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.807 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.807 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.808 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.808 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.808 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.808 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.808 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.809 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.809 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.809 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.810 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.811 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.811 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.811 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.812 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.812 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.812 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.812 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.812 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.812 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.812 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.812 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.813 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.813 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.813 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-04-25T13:29:41.814 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-04-25T13:29:41.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.815 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.815 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-04-25T13:29:41.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.815 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:29:41.815 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.816 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-04-25T13:29:41.816 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-04-25T13:29:41.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.816 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.817 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.817 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-04-25T13:29:41.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.818 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-04-25T13:29:41.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:29:41.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.818 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:41.818 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.818 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.819 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.819 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:29:41.819 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.819 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.819 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:29:41.819 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:29:41.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.820 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.820 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.820 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.820 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:29:41.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.821 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.821 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-04-25T13:29:41.821 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.821 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.822 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.822 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-04-25T13:29:41.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-04-25T13:29:41.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:41.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.822 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-04-25T13:29:41.822 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.823 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.823 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.823 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-04-25T13:29:41.823 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-04-25T13:29:41.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.824 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:41.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.826 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.826 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.827 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.828 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.828 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.828 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.828 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.828 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.828 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.828 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.828 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.829 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.829 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.831 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.831 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.831 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.832 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.832 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.832 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.832 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.832 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.832 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.832 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.833 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.833 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.833 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.833 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.834 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.834 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.835 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-04-25T13:29:41.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-04-25T13:29:41.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.835 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.836 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.836 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.836 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.837 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-04-25T13:29:41.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-04-25T13:29:41.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.837 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-04-25T13:29:41.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.838 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.838 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-04-25T13:29:41.838 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-04-25T13:29:41.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:29:41.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.839 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.840 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.840 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:29:41.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.840 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.841 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2024-04-25T13:29:41.841 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T13:29:41.841 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard", 2024-04-25T13:29:41.841 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2024-04-25T13:29:41.841 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2024-04-25T13:29:41.841 INFO:teuthology.orchestra.run.smithi031.stdout: "restful", 2024-04-25T13:29:41.841 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_schedule" 2024-04-25T13:29:41.841 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.842 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-04-25T13:29:41.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-04-25T13:29:41.842 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.842 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.842 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.842 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:41.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:41.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.843 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.843 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:41.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.843 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.843 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:29:41.843 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.844 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.844 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.844 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.845 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.845 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.845 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.845 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.845 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.845 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.845 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.846 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.846 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.846 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.846 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.846 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.847 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.847 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.847 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.847 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.847 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.848 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.848 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.848 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.849 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.849 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.849 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.849 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.849 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.850 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.850 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.850 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.850 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.851 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.851 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.852 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.852 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-04-25T13:29:41.852 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-04-25T13:29:41.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.852 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.853 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.853 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.853 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.853 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:29:41.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.854 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.854 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-04-25T13:29:41.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-04-25T13:29:41.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.854 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.854 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-04-25T13:29:41.855 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.855 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-04-25T13:29:41.855 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.855 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.855 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-04-25T13:29:41.856 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-04-25T13:29:41.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.856 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.856 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.857 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-04-25T13:29:41.857 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.857 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-04-25T13:29:41.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-04-25T13:29:41.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.857 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.858 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.858 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.858 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-04-25T13:29:41.858 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.859 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-04-25T13:29:41.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-04-25T13:29:41.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.859 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.859 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-04-25T13:29:41.860 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.860 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.860 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.860 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-04-25T13:29:41.860 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.860 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.860 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-04-25T13:29:41.861 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-04-25T13:29:41.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.861 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.861 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.861 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.861 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.861 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:29:41.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.862 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.862 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-04-25T13:29:41.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-04-25T13:29:41.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.863 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:29:41.863 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.864 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.864 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-04-25T13:29:41.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-04-25T13:29:41.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.864 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-04-25T13:29:41.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-04-25T13:29:41.866 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.866 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.866 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.866 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-04-25T13:29:41.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-04-25T13:29:41.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.867 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.867 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.867 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.867 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.867 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:29:41.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.868 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.868 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-04-25T13:29:41.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-04-25T13:29:41.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.869 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-04-25T13:29:41.869 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:29:41.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:29:41.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.870 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-04-25T13:29:41.870 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-04-25T13:29:41.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:41.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:41.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T13:29:41.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T13:29:41.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.871 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:29:41.871 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:29:41.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.871 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-04-25T13:29:41.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:29:41.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:41.872 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-04-25T13:29:41.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:41.872 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-04-25T13:29:41.872 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:29:41.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.873 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.873 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-04-25T13:29:41.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-04-25T13:29:41.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.873 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:29:41.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.874 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:29:41.874 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:29:41.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.874 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.875 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-04-25T13:29:41.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-04-25T13:29:41.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:41.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.875 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-04-25T13:29:41.875 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-04-25T13:29:41.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-04-25T13:29:41.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:29:41.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:29:41.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.876 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-04-25T13:29:41.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-04-25T13:29:41.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-04-25T13:29:41.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.877 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:29:41.877 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:29:41.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.878 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-04-25T13:29:41.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-04-25T13:29:41.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:41.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:41.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-04-25T13:29:41.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-04-25T13:29:41.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:29:41.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:29:41.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.880 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.881 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.881 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.881 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.881 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.881 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.881 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.881 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.881 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.882 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.882 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.884 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.884 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.885 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.885 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.885 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.885 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.885 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.885 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.886 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.886 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.889 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.889 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-04-25T13:29:41.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-04-25T13:29:41.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:41.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:41.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:29:41.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.891 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T13:29:41.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T13:29:41.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.892 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-04-25T13:29:41.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.892 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-04-25T13:29:41.892 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-04-25T13:29:41.892 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-04-25T13:29:41.893 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-04-25T13:29:41.893 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-04-25T13:29:41.893 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-04-25T13:29:41.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.894 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-04-25T13:29:41.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-04-25T13:29:41.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.894 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.895 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.895 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.895 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:29:41.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.895 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.896 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:41.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:41.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.896 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:41.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.897 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:29:41.897 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.897 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.897 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-04-25T13:29:41.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-04-25T13:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-04-25T13:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-04-25T13:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-04-25T13:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-04-25T13:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T13:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-04-25T13:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-04-25T13:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-04-25T13:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-04-25T13:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-04-25T13:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-04-25T13:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-04-25T13:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-04-25T13:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-04-25T13:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-04-25T13:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-04-25T13:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-04-25T13:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-04-25T13:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-04-25T13:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-04-25T13:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-04-25T13:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-04-25T13:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-04-25T13:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-04-25T13:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-04-25T13:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-04-25T13:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-04-25T13:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-04-25T13:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-04-25T13:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-04-25T13:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-04-25T13:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-04-25T13:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-04-25T13:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-04-25T13:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-04-25T13:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-04-25T13:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-04-25T13:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-04-25T13:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-04-25T13:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-04-25T13:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-04-25T13:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-04-25T13:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-04-25T13:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-04-25T13:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-04-25T13:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-04-25T13:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-04-25T13:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-04-25T13:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-04-25T13:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-04-25T13:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-04-25T13:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-04-25T13:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-04-25T13:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-04-25T13:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-04-25T13:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-04-25T13:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-04-25T13:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-04-25T13:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-04-25T13:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-04-25T13:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-04-25T13:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-04-25T13:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-04-25T13:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-04-25T13:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-04-25T13:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-04-25T13:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-04-25T13:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-04-25T13:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-04-25T13:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-04-25T13:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-04-25T13:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-04-25T13:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-04-25T13:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-04-25T13:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-04-25T13:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-04-25T13:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-04-25T13:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-04-25T13:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-04-25T13:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-04-25T13:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-04-25T13:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-04-25T13:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-04-25T13:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-04-25T13:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-04-25T13:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-04-25T13:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-04-25T13:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-04-25T13:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-04-25T13:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-04-25T13:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-04-25T13:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-04-25T13:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-04-25T13:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-04-25T13:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-04-25T13:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-04-25T13:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-04-25T13:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-04-25T13:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-04-25T13:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-04-25T13:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-04-25T13:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-04-25T13:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-04-25T13:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-04-25T13:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-04-25T13:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-04-25T13:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-04-25T13:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-04-25T13:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-04-25T13:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-04-25T13:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-04-25T13:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-04-25T13:29:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-04-25T13:29:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-04-25T13:29:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:29:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-04-25T13:29:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-04-25T13:29:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:29:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-04-25T13:29:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-04-25T13:29:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:29:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-04-25T13:29:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-04-25T13:29:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:29:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-04-25T13:29:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:29:42.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.003 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.003 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.003 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.003 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.003 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:29:42.004 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.004 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.004 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-04-25T13:29:42.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:29:42.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.005 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:29:42.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.005 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.006 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-04-25T13:29:42.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:29:42.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.006 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.006 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.006 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.007 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:29:42.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-04-25T13:29:42.008 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.008 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.008 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.008 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.009 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.009 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.009 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.009 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.010 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.010 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.010 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.010 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.010 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.010 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.010 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.011 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.011 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.011 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.012 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.012 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.013 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.013 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.013 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.013 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.014 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.014 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.014 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.014 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.014 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.015 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.015 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.016 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.016 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.016 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.017 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-04-25T13:29:42.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-04-25T13:29:42.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-04-25T13:29:42.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.017 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:29:42.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.018 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-04-25T13:29:42.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-04-25T13:29:42.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-04-25T13:29:42.019 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:29:42.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.020 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-04-25T13:29:42.021 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.021 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.021 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.021 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:29:42.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:29:42.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:42.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.022 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.023 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:29:42.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:29:42.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.024 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:29:42.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:29:42.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.025 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:29:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:29:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:29:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:29:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:29:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:29:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:29:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:29:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:29:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:29:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:29:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:29:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:29:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:29:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:29:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:29:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:29:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:29:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:29:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:29:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.043 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:29:42.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:29:42.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.043 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.044 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.044 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:29:42.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:29:42.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T13:29:42.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.046 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:29:42.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:29:42.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.048 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-04-25T13:29:42.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:29:42.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.049 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.050 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:29:42.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:29:42.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T13:29:42.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.051 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:29:42.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:29:42.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.052 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.052 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.053 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:29:42.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:29:42.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.053 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.053 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.055 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:29:42.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:29:42.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.056 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.056 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:29:42.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:29:42.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.058 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.058 INFO:teuthology.orchestra.run.smithi031.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:29:42.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:29:42.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "[]", 2024-04-25T13:29:42.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.059 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.060 INFO:teuthology.orchestra.run.smithi031.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:29:42.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:29:42.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "{}", 2024-04-25T13:29:42.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.061 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:29:42.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:29:42.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.063 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:29:42.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:29:42.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.065 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:29:42.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:29:42.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.066 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:29:42.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:29:42.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.067 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.067 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.068 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:29:42.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:29:42.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.070 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:29:42.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:29:42.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.071 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:29:42.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:29:42.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.073 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:29:42.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:29:42.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.073 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.075 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:29:42.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:29:42.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.076 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.076 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.076 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:29:42.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:29:42.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.078 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:29:42.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:29:42.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:29:42.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.079 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.080 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:29:42.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:29:42.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.080 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:42.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.081 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.081 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:29:42.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:29:42.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-04-25T13:29:42.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:29:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:29:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-04-25T13:29:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:29:42.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:29:42.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.085 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.086 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:29:42.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:29:42.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-04-25T13:29:42.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.087 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.088 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:29:42.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:29:42.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.089 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:29:42.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:29:42.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.090 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.091 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:29:42.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:29:42.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.093 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:29:42.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:29:42.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:42.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.094 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.095 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:29:42.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:29:42.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.095 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.095 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:42.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.096 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:29:42.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:29:42.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:42.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.098 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.098 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-04-25T13:29:42.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-04-25T13:29:42.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.099 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.099 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.100 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-04-25T13:29:42.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-04-25T13:29:42.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.100 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.100 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.101 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-04-25T13:29:42.101 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-04-25T13:29:42.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:29:42.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.103 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-04-25T13:29:42.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-04-25T13:29:42.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-04-25T13:29:42.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.105 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T13:29:42.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T13:29:42.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.106 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.107 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.107 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.107 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.108 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.108 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.108 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.108 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.108 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.109 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.110 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.110 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.110 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.111 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.111 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.112 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.112 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.112 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.112 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.112 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.112 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.113 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.115 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-04-25T13:29:42.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-04-25T13:29:42.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-04-25T13:29:42.116 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.116 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:29:42.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:29:42.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.117 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.118 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.118 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T13:29:42.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T13:29:42.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T13:29:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.120 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T13:29:42.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T13:29:42.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-04-25T13:29:42.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.121 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T13:29:42.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T13:29:42.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.123 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-04-25T13:29:42.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-04-25T13:29:42.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-04-25T13:29:42.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.124 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T13:29:42.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T13:29:42.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-04-25T13:29:42.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.125 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.126 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.126 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-04-25T13:29:42.126 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.127 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T13:29:42.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T13:29:42.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T13:29:42.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T13:29:42.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T13:29:42.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.128 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-04-25T13:29:42.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-04-25T13:29:42.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.130 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T13:29:42.130 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.130 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.131 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.131 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-04-25T13:29:42.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-04-25T13:29:42.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.131 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-04-25T13:29:42.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.132 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.133 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.133 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.134 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.134 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.134 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.134 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.134 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.135 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.136 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.138 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.138 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.138 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.138 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.138 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.138 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.138 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.139 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.139 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-04-25T13:29:42.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-04-25T13:29:42.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-04-25T13:29:42.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.142 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:29:42.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.142 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-04-25T13:29:42.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-04-25T13:29:42.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-04-25T13:29:42.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:29:42.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.144 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.144 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-04-25T13:29:42.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-04-25T13:29:42.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-04-25T13:29:42.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.145 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.145 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:29:42.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.146 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-04-25T13:29:42.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-04-25T13:29:42.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T13:29:42.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:29:42.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.148 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-04-25T13:29:42.148 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-04-25T13:29:42.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.148 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.148 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:29:42.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.149 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:42.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:42.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:42.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:29:42.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.151 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.151 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-04-25T13:29:42.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-04-25T13:29:42.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-04-25T13:29:42.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:29:42.152 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-04-25T13:29:42.153 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.153 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.153 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.153 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.155 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.155 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.155 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.155 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.155 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.155 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.155 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.156 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.156 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.157 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.158 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.158 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.159 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.159 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.159 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.159 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.159 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.159 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.159 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.160 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.160 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.162 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-04-25T13:29:42.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-04-25T13:29:42.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-04-25T13:29:42.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.163 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-04-25T13:29:42.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-04-25T13:29:42.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-04-25T13:29:42.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.165 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:42.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:42.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.166 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-04-25T13:29:42.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-04-25T13:29:42.167 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-04-25T13:29:42.167 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:29:42.168 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.168 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-04-25T13:29:42.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-04-25T13:29:42.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-04-25T13:29:42.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:29:42.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.169 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-04-25T13:29:42.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-04-25T13:29:42.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-04-25T13:29:42.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:29:42.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.171 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-04-25T13:29:42.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-04-25T13:29:42.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:29:42.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.173 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:42.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:42.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-04-25T13:29:42.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-04-25T13:29:42.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:29:42.174 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.175 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.176 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.176 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.176 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.176 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.176 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.176 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.176 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.177 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.181 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.182 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.182 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.182 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.182 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.183 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.183 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.183 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.183 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.184 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.185 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-04-25T13:29:42.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-04-25T13:29:42.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:29:42.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.187 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-04-25T13:29:42.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-04-25T13:29:42.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-04-25T13:29:42.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-04-25T13:29:42.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.189 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-04-25T13:29:42.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-04-25T13:29:42.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-04-25T13:29:42.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:29:42.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.190 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-04-25T13:29:42.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-04-25T13:29:42.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:42.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:42.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-04-25T13:29:42.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:29:42.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.192 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-04-25T13:29:42.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-04-25T13:29:42.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:29:42.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.194 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-04-25T13:29:42.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-04-25T13:29:42.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-04-25T13:29:42.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:29:42.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-04-25T13:29:42.196 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.196 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.196 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.196 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.197 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.197 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.197 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.198 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.198 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.198 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.198 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.199 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.200 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.201 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.202 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.202 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.202 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.202 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.202 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.202 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-04-25T13:29:42.205 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.205 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.205 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.205 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.206 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.207 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.207 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.207 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.207 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.207 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.207 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.208 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.209 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-04-25T13:29:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-04-25T13:29:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:29:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-04-25T13:29:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:29:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-04-25T13:29:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-04-25T13:29:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-04-25T13:29:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:29:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.221 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.221 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.221 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.222 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.223 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.223 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.223 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.223 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.223 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.223 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.224 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.224 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.224 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.225 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.225 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.225 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-04-25T13:29:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-04-25T13:29:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-04-25T13:29:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-04-25T13:29:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:29:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:29:42.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.230 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.233 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-04-25T13:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-04-25T13:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:29:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:29:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-04-25T13:29:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-04-25T13:29:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-04-25T13:29:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:29:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-04-25T13:29:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-04-25T13:29:42.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-04-25T13:29:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:29:42.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-04-25T13:29:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-04-25T13:29:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:29:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.244 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-04-25T13:29:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-04-25T13:29:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-04-25T13:29:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:29:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:29:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-04-25T13:29:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.248 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.249 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.252 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.253 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.254 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-04-25T13:29:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.258 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.259 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.262 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-04-25T13:29:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.267 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.272 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T13:29:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-04-25T13:29:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-04-25T13:29:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:29:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.279 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.283 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-04-25T13:29:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-04-25T13:29:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-04-25T13:29:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-04-25T13:29:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-04-25T13:29:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-04-25T13:29:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-04-25T13:29:42.288 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.290 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.294 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.295 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-04-25T13:29:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.299 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.300 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.303 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-04-25T13:29:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.307 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.308 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.312 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.315 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:42.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:42.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:42.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.317 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-04-25T13:29:42.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-04-25T13:29:42.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:42.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-04-25T13:29:42.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-04-25T13:29:42.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-04-25T13:29:42.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:29:42.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-04-25T13:29:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-04-25T13:29:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:29:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:29:42.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T13:29:42.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T13:29:42.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.322 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.323 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.324 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.324 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.324 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.324 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.324 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.324 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.325 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.325 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.327 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.328 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.328 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.328 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.328 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.328 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.329 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.329 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.329 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-04-25T13:29:42.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-04-25T13:29:42.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-04-25T13:29:42.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:29:42.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.333 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-04-25T13:29:42.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-04-25T13:29:42.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-04-25T13:29:42.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:29:42.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-04-25T13:29:42.335 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.335 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.335 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.335 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-04-25T13:29:42.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-04-25T13:29:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.337 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-04-25T13:29:42.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-04-25T13:29:42.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:29:42.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:29:42.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.339 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.340 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.340 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.340 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.340 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.340 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.340 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.340 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.341 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.343 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.344 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.344 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.344 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.344 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.344 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.344 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.344 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.345 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.347 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-04-25T13:29:42.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-04-25T13:29:42.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.349 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:29:42.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:29:42.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-04-25T13:29:42.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.350 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-04-25T13:29:42.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-04-25T13:29:42.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:42.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-04-25T13:29:42.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T13:29:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T13:29:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-04-25T13:29:42.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:29:42.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.354 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T13:29:42.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T13:29:42.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-04-25T13:29:42.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:29:42.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.355 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-04-25T13:29:42.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-04-25T13:29:42.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-04-25T13:29:42.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.357 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-04-25T13:29:42.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-04-25T13:29:42.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-04-25T13:29:42.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-04-25T13:29:42.358 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-04-25T13:29:42.358 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.359 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-04-25T13:29:42.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-04-25T13:29:42.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-04-25T13:29:42.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-04-25T13:29:42.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-04-25T13:29:42.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T13:29:42.361 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.361 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.361 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.362 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.363 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.363 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.363 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.363 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.363 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.363 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.363 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.364 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.366 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.367 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.367 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.367 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.367 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.367 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.367 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.368 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.368 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-04-25T13:29:42.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:29:42.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-04-25T13:29:42.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.372 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:29:42.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:29:42.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.373 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-04-25T13:29:42.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-04-25T13:29:42.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-04-25T13:29:42.376 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.376 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.376 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-04-25T13:29:42.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-04-25T13:29:42.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.378 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-04-25T13:29:42.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-04-25T13:29:42.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.378 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.379 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.380 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.380 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.381 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.381 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.381 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.381 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.381 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.382 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.383 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.384 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.385 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.385 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.385 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.385 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.385 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.385 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.386 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.387 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-04-25T13:29:42.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-04-25T13:29:42.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.389 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-04-25T13:29:42.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-04-25T13:29:42.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-04-25T13:29:42.391 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.391 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.391 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.393 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.393 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.393 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.393 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.393 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.393 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.393 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.394 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.396 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.397 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.397 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.397 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.397 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.397 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.397 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.397 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.398 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-04-25T13:29:42.400 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.400 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.400 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.401 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.402 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.402 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.402 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.402 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.402 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.402 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.402 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.403 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.405 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.406 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.406 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.406 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.406 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.406 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.406 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.407 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.407 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.409 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-04-25T13:29:42.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-04-25T13:29:42.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-04-25T13:29:42.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:29:42.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-04-25T13:29:42.411 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.411 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.411 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.411 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.412 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.413 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.413 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.413 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.413 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.413 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.413 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.414 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.416 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.416 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.417 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.417 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.417 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.417 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.417 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.417 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.417 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.418 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.420 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-04-25T13:29:42.420 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-04-25T13:29:42.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.421 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.421 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.421 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-04-25T13:29:42.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-04-25T13:29:42.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.422 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.422 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-04-25T13:29:42.422 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.423 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-04-25T13:29:42.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-04-25T13:29:42.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.424 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.424 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.424 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.425 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-04-25T13:29:42.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-04-25T13:29:42.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.425 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.426 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-04-25T13:29:42.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-04-25T13:29:42.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-04-25T13:29:42.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.428 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-04-25T13:29:42.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-04-25T13:29:42.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.430 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-04-25T13:29:42.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-04-25T13:29:42.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.431 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-04-25T13:29:42.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-04-25T13:29:42.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.433 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-04-25T13:29:42.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-04-25T13:29:42.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-04-25T13:29:42.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-04-25T13:29:42.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.435 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-04-25T13:29:42.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-04-25T13:29:42.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.436 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.436 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-04-25T13:29:42.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-04-25T13:29:42.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.437 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.437 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.437 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.438 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.438 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-04-25T13:29:42.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-04-25T13:29:42.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-04-25T13:29:42.440 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.440 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.440 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.440 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-04-25T13:29:42.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-04-25T13:29:42.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.441 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.441 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:29:42.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.442 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.442 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-04-25T13:29:42.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-04-25T13:29:42.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.443 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.443 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:29:42.443 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.444 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.444 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.445 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.445 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.445 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.445 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.445 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.445 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.445 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.446 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.448 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.449 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.449 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.449 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.449 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.449 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.449 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.449 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.450 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.451 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-04-25T13:29:42.452 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.452 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.452 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.452 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.454 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.454 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.454 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.454 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.454 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.454 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.454 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.455 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.457 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.458 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.458 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.458 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.458 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.458 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.458 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.459 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.459 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.460 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.461 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-04-25T13:29:42.461 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.462 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.462 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.462 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.463 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.463 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.463 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.463 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.463 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.464 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.464 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.464 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.466 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.467 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.467 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.468 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.468 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.468 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.468 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.468 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.469 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.470 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-04-25T13:29:42.473 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.473 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.473 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.474 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-04-25T13:29:42.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-04-25T13:29:42.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.474 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:29:42.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.475 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.475 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:42.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:42.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-04-25T13:29:42.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.477 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.478 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.478 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.478 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.478 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.478 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.478 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.478 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.478 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.479 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.479 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.479 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.481 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.482 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.482 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.482 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.482 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.483 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.483 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.483 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.483 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.483 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.483 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.483 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.484 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-04-25T13:29:42.486 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.486 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.486 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.486 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-04-25T13:29:42.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-04-25T13:29:42.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.487 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.487 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.487 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:29:42.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.488 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-04-25T13:29:42.488 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-04-25T13:29:42.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.489 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:29:42.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.490 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-04-25T13:29:42.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-04-25T13:29:42.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:29:42.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.491 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-04-25T13:29:42.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-04-25T13:29:42.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.492 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.492 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:29:42.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.493 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-04-25T13:29:42.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-04-25T13:29:42.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.494 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:29:42.494 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.495 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-04-25T13:29:42.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-04-25T13:29:42.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.495 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.496 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.496 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.496 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.497 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-04-25T13:29:42.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-04-25T13:29:42.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.497 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.498 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.498 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.498 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-04-25T13:29:42.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-04-25T13:29:42.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:29:42.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-04-25T13:29:42.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-04-25T13:29:42.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.501 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.501 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:42.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:42.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-04-25T13:29:42.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-04-25T13:29:42.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.503 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.503 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-04-25T13:29:42.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-04-25T13:29:42.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-04-25T13:29:42.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.505 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-04-25T13:29:42.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-04-25T13:29:42.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.506 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.506 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.507 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-04-25T13:29:42.507 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-04-25T13:29:42.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.508 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.508 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.508 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.508 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.509 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.509 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.512 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.512 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.513 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.513 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.514 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.514 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.514 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.514 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.514 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.514 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.514 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.515 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.515 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.516 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.516 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-04-25T13:29:42.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-04-25T13:29:42.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.517 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.517 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.518 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-04-25T13:29:42.518 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-04-25T13:29:42.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.519 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.519 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.520 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-04-25T13:29:42.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-04-25T13:29:42.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.520 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:29:42.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.521 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-04-25T13:29:42.522 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.522 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.522 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.522 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.523 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.523 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.523 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.524 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.524 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.524 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.524 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.524 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.525 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.526 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.527 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.527 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.528 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.528 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.528 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.528 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.528 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.528 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.529 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.529 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.529 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.530 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T13:29:42.531 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.531 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.531 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.531 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.532 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.532 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.532 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.532 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.532 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.533 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.533 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.533 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.534 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.534 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.535 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.535 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.536 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.536 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.536 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.537 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.537 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.537 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.537 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.538 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.539 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.539 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-04-25T13:29:42.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-04-25T13:29:42.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.540 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-04-25T13:29:42.540 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.540 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:29:42.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.541 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-04-25T13:29:42.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-04-25T13:29:42.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.542 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-04-25T13:29:42.542 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.542 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-04-25T13:29:42.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:29:42.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-04-25T13:29:42.543 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.543 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:29:42.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.544 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:29:42.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:29:42.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-04-25T13:29:42.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:29:42.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.546 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-04-25T13:29:42.546 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-04-25T13:29:42.546 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-04-25T13:29:42.547 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-04-25T13:29:42.547 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-04-25T13:29:42.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-04-25T13:29:42.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-04-25T13:29:42.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-04-25T13:29:42.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.548 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.548 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.548 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.548 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.548 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-04-25T13:29:42.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-04-25T13:29:42.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.549 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.549 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.549 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.549 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.550 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.550 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.550 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-04-25T13:29:42.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-04-25T13:29:42.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-04-25T13:29:42.550 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.551 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-04-25T13:29:42.551 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.551 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.552 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-04-25T13:29:42.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-04-25T13:29:42.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.553 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.553 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.553 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.553 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.553 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.553 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.553 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.554 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.554 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.554 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.554 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-04-25T13:29:42.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-04-25T13:29:42.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.555 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.555 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.555 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.555 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.556 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.556 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-04-25T13:29:42.556 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-04-25T13:29:42.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.556 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-04-25T13:29:42.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-04-25T13:29:42.557 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-04-25T13:29:42.557 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-04-25T13:29:42.557 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-04-25T13:29:42.557 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-04-25T13:29:42.557 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-04-25T13:29:42.558 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-04-25T13:29:42.558 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.558 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.558 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.558 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.558 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-04-25T13:29:42.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-04-25T13:29:42.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-04-25T13:29:42.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-04-25T13:29:42.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-04-25T13:29:42.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.559 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.559 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.559 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.560 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.560 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-04-25T13:29:42.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-04-25T13:29:42.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.561 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-04-25T13:29:42.561 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.561 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-04-25T13:29:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-04-25T13:29:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-04-25T13:29:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-04-25T13:29:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-04-25T13:29:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-04-25T13:29:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-04-25T13:29:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-04-25T13:29:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-04-25T13:29:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:29:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-04-25T13:29:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-04-25T13:29:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-04-25T13:29:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-04-25T13:29:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-04-25T13:29:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-04-25T13:29:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-04-25T13:29:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "services": { 2024-04-25T13:29:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard": "https://172.21.15.31:8443/" 2024-04-25T13:29:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2024-04-25T13:29:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2024-04-25T13:29:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T13:29:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T13:29:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T13:29:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T13:29:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T13:29:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T13:29:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T13:29:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T13:29:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T13:29:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T13:29:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2024-04-25T13:29:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T13:29:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T13:29:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T13:29:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T13:29:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T13:29:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T13:29:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T13:29:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T13:29:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T13:29:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T13:29:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2024-04-25T13:29:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T13:29:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T13:29:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T13:29:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T13:29:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T13:29:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T13:29:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T13:29:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T13:29:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T13:29:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T13:29:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "reef": [ 2024-04-25T13:29:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T13:29:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T13:29:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T13:29:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T13:29:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T13:29:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T13:29:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T13:29:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T13:29:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T13:29:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T13:29:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-25T13:29:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "squid": [ 2024-04-25T13:29:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-04-25T13:29:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-25T13:29:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-04-25T13:29:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-04-25T13:29:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-04-25T13:29:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-04-25T13:29:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-04-25T13:29:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-04-25T13:29:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-04-25T13:29:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-04-25T13:29:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:29:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 91, 2024-04-25T13:29:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2024-04-25T13:29:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-04-25T13:29:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T13:29:42.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T13:29:42.574 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T13:29:42.574 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1688182418 2024-04-25T13:29:42.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.574 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:29:42.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2024-04-25T13:29:42.575 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T13:29:42.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T13:29:42.575 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T13:29:42.575 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3361623760 2024-04-25T13:29:42.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:29:42.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-04-25T13:29:42.576 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T13:29:42.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T13:29:42.576 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T13:29:42.576 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3379920468 2024-04-25T13:29:42.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.577 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:29:42.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.577 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-04-25T13:29:42.577 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T13:29:42.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T13:29:42.577 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T13:29:42.578 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2401718411 2024-04-25T13:29:42.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.578 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:29:42.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-04-25T13:29:42.578 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T13:29:42.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T13:29:42.579 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T13:29:42.579 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 4166165552 2024-04-25T13:29:42.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:29:42.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:42.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-04-25T13:29:42.579 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-04-25T13:29:42.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-04-25T13:29:42.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-04-25T13:29:42.580 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-04-25T13:29:42.580 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1402742832 2024-04-25T13:29:42.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:29:42.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:42.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-25T13:29:42.581 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T13:29:42.582 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:40 smithi136 ceph-mon[33038]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:29:42.582 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:40 smithi136 ceph-mon[33038]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:29:42.583 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:40 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3862946163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:29:42.583 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:40 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2273720927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:29:42.583 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:40 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/4241399010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:29:42.583 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:41 smithi136 ceph-mon[33038]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:42.583 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:41 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3633910355' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:29:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:40 smithi066 ceph-mon[33137]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:29:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:40 smithi066 ceph-mon[33137]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:29:42.583 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:40 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3862946163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:29:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:40 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2273720927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:29:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:40 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/4241399010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:29:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:41 smithi066 ceph-mon[33137]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:42.584 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:41 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3633910355' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:29:42.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.371+0000 7fd8498a7640 1 -- 172.21.15.31:0/3633910355 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd82c0a3d20 msgr2=0x7fd82c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:42.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.371+0000 7fd8498a7640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd82c0a3d20 0x7fd82c0a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd83400bbe0 tx=0x7fd83400b390 comp rx=0 tx=0).stop 2024-04-25T13:29:42.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.371+0000 7fd8498a7640 1 -- 172.21.15.31:0/3633910355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd844103b50 msgr2=0x7fd844198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:29:42.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.371+0000 7fd8498a7640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd844103b50 0x7fd844198c30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c890 tx=0x7fd83800cd50 comp rx=0 tx=0).stop 2024-04-25T13:29:42.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.371+0000 7fd8498a7640 1 -- 172.21.15.31:0/3633910355 shutdown_connections 2024-04-25T13:29:42.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.371+0000 7fd8498a7640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd82c0a3d20 0x7fd82c0a61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:42.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.372+0000 7fd8498a7640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84410c570 0x7fd84419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:42.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.372+0000 7fd8498a7640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd844103b50 0x7fd844198c30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:42.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.372+0000 7fd8498a7640 1 --2- 172.21.15.31:0/3633910355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd844103230 0x7fd8441986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:29:42.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.372+0000 7fd8498a7640 1 -- 172.21.15.31:0/3633910355 >> 172.21.15.31:0/3633910355 conn(0x7fd844077fb0 msgr2=0x7fd8440fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:29:42.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.372+0000 7fd8498a7640 1 -- 172.21.15.31:0/3633910355 shutdown_connections 2024-04-25T13:29:42.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:29:40.372+0000 7fd8498a7640 1 -- 172.21.15.31:0/3633910355 wait complete. 2024-04-25T13:29:42.586 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:40 smithi031 ceph-mon[29457]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:29:42.586 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:40 smithi031 ceph-mon[29457]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:29:42.586 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:40 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3862946163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:29:42.586 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:40 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2273720927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:29:42.586 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:40 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/4241399010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:29:42.598 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T13:29:42.598 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T13:29:42.599 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:41 smithi031 ceph-mon[29457]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:42.599 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:41 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3633910355' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:29:42.860 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:29:43.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:43 smithi031 ceph-mon[29457]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:43.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:43 smithi066 ceph-mon[33137]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:43.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:43 smithi136 ceph-mon[33038]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T13:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-04-25T13:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-04-25T13:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-04-25T13:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-04-25T13:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-04-25T13:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.001489389, 2024-04-25T13:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000342371, 2024-04-25T13:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 212, 2024-04-25T13:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 199, 2024-04-25T13:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1588784, 2024-04-25T13:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 121312, 2024-04-25T13:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-04-25T13:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-04-25T13:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.043606808, 2024-04-25T13:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.013665023, 2024-04-25T13:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.033781013, 2024-04-25T13:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.004272049, 2024-04-25T13:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 199, 2024-04-25T13:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.011894283, 2024-04-25T13:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000059770 2024-04-25T13:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1588784, 2024-04-25T13:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 121312 2024-04-25T13:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2, 2024-04-25T13:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 150, 2024-04-25T13:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 182384, 2024-04-25T13:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2024-04-25T13:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2024-04-25T13:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.013536186, 2024-04-25T13:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.008947655, 2024-04-25T13:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002148701, 2024-04-25T13:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 147, 2024-04-25T13:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.017159686, 2024-04-25T13:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000116732 2024-04-25T13:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 182384 2024-04-25T13:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T13:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-04-25T13:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T13:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T13:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T13:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T13:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T13:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T13:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T13:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T13:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T13:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2024-04-25T13:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T13:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T13:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T13:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T13:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T13:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-04-25T13:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-04-25T13:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T13:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T13:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-04-25T13:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-04-25T13:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T13:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T13:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T13:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T13:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T13:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T13:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T13:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 210, 2024-04-25T13:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1556540, 2024-04-25T13:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T13:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 210, 2024-04-25T13:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T13:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T13:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 210, 2024-04-25T13:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1556540, 2024-04-25T13:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T13:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T13:29:45.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:45 smithi066 ceph-mon[33137]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:45.422 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:45 smithi031 ceph-mon[29457]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:45.454 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T13:29:45.454 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T13:29:45.455 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T13:29:45.455 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T13:29:45.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T13:29:45.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:45 smithi136 ceph-mon[33038]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:29:47.207 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:47 smithi031 ceph-mon[29457]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:47.340 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T13:29:47.340 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:29:47.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-04-25T13:29:47.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-04-25T13:29:47.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:29:47.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-04-25T13:29:47.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-04-25T13:29:47.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-04-25T13:29:47.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.001363795, 2024-04-25T13:29:47.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:29:47.341 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000328239, 2024-04-25T13:29:47.342 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:29:47.342 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:47.342 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:47.342 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:47.342 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:47.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:47.342 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:47.342 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:47.343 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:47.343 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:47.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:47.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:29:47.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:29:47.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:47.343 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:29:47.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 218, 2024-04-25T13:29:47.343 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 205, 2024-04-25T13:29:47.344 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1590416, 2024-04-25T13:29:47.344 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 125152, 2024-04-25T13:29:47.344 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-04-25T13:29:47.344 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-04-25T13:29:47.344 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.046002957, 2024-04-25T13:29:47.344 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.013954744, 2024-04-25T13:29:47.344 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.038209254, 2024-04-25T13:29:47.345 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.004295939, 2024-04-25T13:29:47.345 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:47.345 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 205, 2024-04-25T13:29:47.345 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.011223035, 2024-04-25T13:29:47.345 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000054746 2024-04-25T13:29:47.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:47.345 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:47.345 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:47.346 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:47.346 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:47.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:47.346 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1590416, 2024-04-25T13:29:47.346 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 125152 2024-04-25T13:29:47.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:47.346 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:29:47.346 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2, 2024-04-25T13:29:47.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 155, 2024-04-25T13:29:47.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:29:47.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 183264, 2024-04-25T13:29:47.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2024-04-25T13:29:47.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2024-04-25T13:29:47.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.014046124, 2024-04-25T13:29:47.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.009070979, 2024-04-25T13:29:47.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001876645, 2024-04-25T13:29:47.347 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:29:47.348 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:47.348 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 152, 2024-04-25T13:29:47.348 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.019309968, 2024-04-25T13:29:47.348 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000127039 2024-04-25T13:29:47.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:47.348 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:47.348 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:47.348 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:47.348 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:47.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:47.349 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:29:47.349 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 183264 2024-04-25T13:29:47.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:47.349 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T13:29:47.349 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-04-25T13:29:47.349 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T13:29:47.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:47.350 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T13:29:47.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:29:47.350 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T13:29:47.351 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:29:47.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:29:47.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:29:47.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:29:47.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:29:47.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:29:47.352 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:29:47.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:29:47.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:29:47.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:29:47.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:29:47.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:29:47.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:29:47.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T13:29:47.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:29:47.353 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T13:29:47.354 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:29:47.354 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:29:47.354 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:29:47.354 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:29:47.354 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:29:47.354 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:29:47.354 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:29:47.354 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T13:29:47.355 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:29:47.355 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:29:47.355 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:29:47.355 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T13:29:47.355 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T13:29:47.355 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T13:29:47.355 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:29:47.355 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:29:47.355 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:29:47.356 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:29:47.356 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 149034, 2024-04-25T13:29:47.356 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 32, 2024-04-25T13:29:47.356 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:29:47.356 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T13:29:47.356 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T13:29:47.356 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T13:29:47.356 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:29:47.357 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T13:29:47.357 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:29:47.357 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T13:29:47.357 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-04-25T13:29:47.357 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-04-25T13:29:47.357 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:29:47.357 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:29:47.357 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T13:29:47.357 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T13:29:47.358 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-04-25T13:29:47.358 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-04-25T13:29:47.358 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T13:29:47.358 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T13:29:47.358 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T13:29:47.358 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T13:29:47.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:47.358 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:29:47.359 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T13:29:47.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T13:29:47.359 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T13:29:47.359 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 216, 2024-04-25T13:29:47.359 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1557320, 2024-04-25T13:29:47.359 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T13:29:47.359 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 216, 2024-04-25T13:29:47.359 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T13:29:47.359 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T13:29:47.360 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 216, 2024-04-25T13:29:47.360 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1557320, 2024-04-25T13:29:47.360 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T13:29:47.360 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:47.360 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:47.360 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:47.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:47.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:47.361 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T13:29:47.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:47 smithi066 ceph-mon[33137]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:47.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:47 smithi136 ceph-mon[33038]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:47.955 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T13:29:47.956 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T13:29:47.956 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T13:29:47.956 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T13:29:47.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T13:29:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:29:49.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:49 smithi066 ceph-mon[33137]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:49.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:49 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:29:49.434 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:49 smithi031 ceph-mon[29457]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:49.434 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:49 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:29:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:49 smithi136 ceph-mon[33038]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:49.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:49 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T13:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 214, 2024-04-25T13:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 201, 2024-04-25T13:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1589328, 2024-04-25T13:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 122592, 2024-04-25T13:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-04-25T13:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-04-25T13:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.044026785, 2024-04-25T13:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.013773899, 2024-04-25T13:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.034836350, 2024-04-25T13:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.004205513, 2024-04-25T13:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 201, 2024-04-25T13:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.011715301, 2024-04-25T13:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000058285 2024-04-25T13:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1589328, 2024-04-25T13:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 122592 2024-04-25T13:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2, 2024-04-25T13:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 152, 2024-04-25T13:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 182736, 2024-04-25T13:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2024-04-25T13:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2024-04-25T13:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.013100280, 2024-04-25T13:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.008883227, 2024-04-25T13:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001853049, 2024-04-25T13:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 149, 2024-04-25T13:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.017081642, 2024-04-25T13:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000114641 2024-04-25T13:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 182736 2024-04-25T13:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-04-25T13:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-04-25T13:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-04-25T13:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-04-25T13:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-04-25T13:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.000287099, 2024-04-25T13:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000210636, 2024-04-25T13:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T13:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-04-25T13:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T13:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T13:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T13:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T13:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T13:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T13:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T13:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T13:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T13:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 149011, 2024-04-25T13:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 32, 2024-04-25T13:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T13:29:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T13:29:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T13:29:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:29:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T13:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T13:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-04-25T13:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-04-25T13:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T13:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T13:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-04-25T13:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-04-25T13:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T13:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T13:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T13:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T13:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T13:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T13:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T13:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 212, 2024-04-25T13:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1556800, 2024-04-25T13:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T13:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 212, 2024-04-25T13:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T13:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T13:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 212, 2024-04-25T13:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1556800, 2024-04-25T13:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T13:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T13:29:51.177 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:51.178 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.178 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.178 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.178 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.178 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.178 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:51 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.204 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T13:29:51.205 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T13:29:51.205 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T13:29:51.205 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T13:29:51.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T13:29:51.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:51.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.410 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:51 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:51.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:51.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:51 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:29:52.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:52 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T13:29:52.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:52 smithi031 ceph-mon[29457]: Unable to set osd_memory_target on smithi136 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:29:52.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:52 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-25T13:29:52.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:52 smithi031 ceph-mon[29457]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T13:29:52.235 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:52 smithi031 ceph-mon[29457]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:52.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:52 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T13:29:52.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:52 smithi066 ceph-mon[33137]: Unable to set osd_memory_target on smithi136 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:29:52.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:52 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-25T13:29:52.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:52 smithi066 ceph-mon[33137]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T13:29:52.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:52 smithi066 ceph-mon[33137]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:52.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:52 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T13:29:52.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:52 smithi136 ceph-mon[33038]: Unable to set osd_memory_target on smithi136 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T13:29:52.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:52 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi066 to 176.2M 2024-04-25T13:29:52.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:52 smithi136 ceph-mon[33038]: Unable to set osd_memory_target on smithi066 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T13:29:52.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:52 smithi136 ceph-mon[33038]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:29:53.213 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-25T13:29:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:29:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-04-25T13:29:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-04-25T13:29:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:29:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-04-25T13:29:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-04-25T13:29:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-04-25T13:29:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.001285027, 2024-04-25T13:29:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:29:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000313185, 2024-04-25T13:29:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:29:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:29:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:29:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:29:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 216, 2024-04-25T13:29:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 205, 2024-04-25T13:29:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1588016, 2024-04-25T13:29:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 125984, 2024-04-25T13:29:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-04-25T13:29:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-04-25T13:29:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.046289195, 2024-04-25T13:29:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.014280693, 2024-04-25T13:29:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.037117730, 2024-04-25T13:29:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.004390954, 2024-04-25T13:29:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 204, 2024-04-25T13:29:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.012152065, 2024-04-25T13:29:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000059568 2024-04-25T13:29:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1588016, 2024-04-25T13:29:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 125984 2024-04-25T13:29:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:29:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 6, 2024-04-25T13:29:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 159, 2024-04-25T13:29:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 3264, 2024-04-25T13:29:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 184064, 2024-04-25T13:29:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2024-04-25T13:29:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2024-04-25T13:29:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.014304113, 2024-04-25T13:29:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.009160106, 2024-04-25T13:29:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002648336, 2024-04-25T13:29:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:29:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 157, 2024-04-25T13:29:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.017743196, 2024-04-25T13:29:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000113013 2024-04-25T13:29:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T13:29:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 184064 2024-04-25T13:29:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-04-25T13:29:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-04-25T13:29:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-04-25T13:29:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-04-25T13:29:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:29:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-04-25T13:29:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:29:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:29:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:29:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:29:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:29:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:29:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:29:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:29:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:29:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:29:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:29:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:29:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:29:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-04-25T13:29:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:29:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-04-25T13:29:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:29:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:29:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:29:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:29:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:29:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:29:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:29:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-04-25T13:29:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:29:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:29:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:29:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-04-25T13:29:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-04-25T13:29:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-04-25T13:29:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:29:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:29:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:29:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:29:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:29:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2024-04-25T13:29:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:29:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-04-25T13:29:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-04-25T13:29:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-04-25T13:29:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:29:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-04-25T13:29:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:29:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-04-25T13:29:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-04-25T13:29:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-04-25T13:29:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:29:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:29:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-04-25T13:29:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-04-25T13:29:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2024-04-25T13:29:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-04-25T13:29:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-04-25T13:29:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-04-25T13:29:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-04-25T13:29:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-04-25T13:29:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-25T13:29:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:29:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-04-25T13:29:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-04-25T13:29:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-04-25T13:29:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 218, 2024-04-25T13:29:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1557580, 2024-04-25T13:29:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-04-25T13:29:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 218, 2024-04-25T13:29:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-04-25T13:29:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-04-25T13:29:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 218, 2024-04-25T13:29:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1557580, 2024-04-25T13:29:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-04-25T13:29:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-04-25T13:29:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-04-25T13:29:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-04-25T13:29:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-25T13:29:53.248 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-25T13:29:53.834 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T13:29:53.834 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T13:29:53.834 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T13:29:53.835 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T13:29:53.835 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T13:29:53.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:53 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:53.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:53 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:53.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:53 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:53.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:53 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:53.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:53 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:53.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:53 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:53.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:53 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:29:53.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:53 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:29:53.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:53 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:54.096 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:29:54.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:53 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:54.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:53 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:54.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:53 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:54.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:53 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:54.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:53 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:54.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:53 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:54.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:53 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:29:54.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:53 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:29:54.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:53 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:54.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:53 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:54.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:53 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:54.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:53 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:54.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:53 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:54.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:53 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:54.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:53 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:29:54.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:53 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:29:54.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:53 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:29:54.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:53 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' 2024-04-25T13:29:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:54 smithi066 ceph-mon[33137]: Adjusting osd_memory_target on smithi031 to 177.2M 2024-04-25T13:29:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:54 smithi066 ceph-mon[33137]: Unable to set osd_memory_target on smithi031 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:29:54.908 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:54 smithi066 ceph-mon[33137]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:54.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:54 smithi031 ceph-mon[29457]: Adjusting osd_memory_target on smithi031 to 177.2M 2024-04-25T13:29:54.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:54 smithi031 ceph-mon[29457]: Unable to set osd_memory_target on smithi031 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:29:54.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:54 smithi031 ceph-mon[29457]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:55.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:54 smithi136 ceph-mon[33038]: Adjusting osd_memory_target on smithi031 to 177.2M 2024-04-25T13:29:55.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:54 smithi136 ceph-mon[33038]: Unable to set osd_memory_target on smithi031 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:29:55.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:54 smithi136 ceph-mon[33038]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:55.966 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-25T13:29:55.967 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:29:55.967 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 1741, 2024-04-25T13:29:55.967 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 1729, 2024-04-25T13:29:55.967 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2147208, 2024-04-25T13:29:55.967 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 605980, 2024-04-25T13:29:55.967 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-04-25T13:29:55.968 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2024-04-25T13:29:55.968 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.177412820, 2024-04-25T13:29:55.968 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.065609428, 2024-04-25T13:29:55.968 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.064746874, 2024-04-25T13:29:55.968 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.042733305, 2024-04-25T13:29:55.968 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:55.968 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 1728, 2024-04-25T13:29:55.968 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.069636952, 2024-04-25T13:29:55.969 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000040299 2024-04-25T13:29:55.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.969 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:55.969 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.969 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.969 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.970 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1608432, 2024-04-25T13:29:55.970 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 126432 2024-04-25T13:29:55.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.970 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:29:55.970 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 6, 2024-04-25T13:29:55.970 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 156, 2024-04-25T13:29:55.970 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1976, 2024-04-25T13:29:55.971 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 436954, 2024-04-25T13:29:55.971 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 7, 2024-04-25T13:29:55.971 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 5, 2024-04-25T13:29:55.971 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.010310692, 2024-04-25T13:29:55.971 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.006032785, 2024-04-25T13:29:55.971 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.002308749, 2024-04-25T13:29:55.971 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000081030, 2024-04-25T13:29:55.971 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:55.972 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 153, 2024-04-25T13:29:55.972 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.013805083, 2024-04-25T13:29:55.972 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000090229 2024-04-25T13:29:55.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.972 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:55.972 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.972 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.972 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.973 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:29:55.973 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 435616 2024-04-25T13:29:55.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.973 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:29:55.973 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 720, 2024-04-25T13:29:55.973 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 720, 2024-04-25T13:29:55.973 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 179186, 2024-04-25T13:29:55.973 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 228143, 2024-04-25T13:29:55.974 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-04-25T13:29:55.974 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-04-25T13:29:55.974 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.062435597, 2024-04-25T13:29:55.974 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.017331736, 2024-04-25T13:29:55.974 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.014851917, 2024-04-25T13:29:55.974 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.024293045, 2024-04-25T13:29:55.974 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:55.975 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 719, 2024-04-25T13:29:55.975 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.022569843, 2024-04-25T13:29:55.975 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000031390 2024-04-25T13:29:55.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.975 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:55.975 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.975 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.975 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.976 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T13:29:55.976 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:29:55.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.976 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-04-25T13:29:55.976 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 1, 2024-04-25T13:29:55.976 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-04-25T13:29:55.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.976 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-MDSRank": { 2024-04-25T13:29:55.977 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-04-25T13:29:55.977 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-04-25T13:29:55.977 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3732, 2024-04-25T13:29:55.977 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.094780320, 2024-04-25T13:29:55.977 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000025396 2024-04-25T13:29:55.977 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:29:55.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.977 INFO:teuthology.orchestra.run.smithi066.stdout: "finisher-PurgeQueue": { 2024-04-25T13:29:55.977 INFO:teuthology.orchestra.run.smithi066.stdout: "queue_len": 0, 2024-04-25T13:29:55.978 INFO:teuthology.orchestra.run.smithi066.stdout: "complete_latency": { 2024-04-25T13:29:55.978 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.978 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.978 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.979 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:29:55.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.979 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": { 2024-04-25T13:29:55.979 INFO:teuthology.orchestra.run.smithi066.stdout: "request": 0, 2024-04-25T13:29:55.979 INFO:teuthology.orchestra.run.smithi066.stdout: "reply": 0, 2024-04-25T13:29:55.979 INFO:teuthology.orchestra.run.smithi066.stdout: "reply_latency": { 2024-04-25T13:29:55.979 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.979 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.979 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.980 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_reply": 0, 2024-04-25T13:29:55.980 INFO:teuthology.orchestra.run.smithi066.stdout: "forward": 0, 2024-04-25T13:29:55.980 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_complete": 0, 2024-04-25T13:29:55.980 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_fetch_keys": 0, 2024-04-25T13:29:55.980 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_commit": 0, 2024-04-25T13:29:55.980 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_split": 0, 2024-04-25T13:29:55.980 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_merge": 0, 2024-04-25T13:29:55.981 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes": 12, 2024-04-25T13:29:55.981 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_top": 0, 2024-04-25T13:29:55.981 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_bottom": 12, 2024-04-25T13:29:55.981 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pin_tail": 0, 2024-04-25T13:29:55.981 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_pinned": 12, 2024-04-25T13:29:55.981 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_expired": 0, 2024-04-25T13:29:55.981 INFO:teuthology.orchestra.run.smithi066.stdout: "inodes_with_caps": 0, 2024-04-25T13:29:55.981 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": 0, 2024-04-25T13:29:55.981 INFO:teuthology.orchestra.run.smithi066.stdout: "subtrees": 2, 2024-04-25T13:29:55.982 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse": 0, 2024-04-25T13:29:55.982 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_hit": 0, 2024-04-25T13:29:55.982 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_forward": 0, 2024-04-25T13:29:55.982 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_discover": 0, 2024-04-25T13:29:55.982 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:29:55.982 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_remote_ino": 0, 2024-04-25T13:29:55.982 INFO:teuthology.orchestra.run.smithi066.stdout: "traverse_lock": 0, 2024-04-25T13:29:55.982 INFO:teuthology.orchestra.run.smithi066.stdout: "load_cent": 0, 2024-04-25T13:29:55.982 INFO:teuthology.orchestra.run.smithi066.stdout: "q": 0, 2024-04-25T13:29:55.983 INFO:teuthology.orchestra.run.smithi066.stdout: "exported": 0, 2024-04-25T13:29:55.983 INFO:teuthology.orchestra.run.smithi066.stdout: "exported_inodes": 0, 2024-04-25T13:29:55.983 INFO:teuthology.orchestra.run.smithi066.stdout: "imported": 0, 2024-04-25T13:29:55.983 INFO:teuthology.orchestra.run.smithi066.stdout: "imported_inodes": 0, 2024-04-25T13:29:55.983 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_dir_fetch": 0, 2024-04-25T13:29:55.983 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:29:55.983 INFO:teuthology.orchestra.run.smithi066.stdout: "openino_peer_discover": 0, 2024-04-25T13:29:55.983 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rfiles": 4, 2024-04-25T13:29:55.983 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rbytes": 234, 2024-04-25T13:29:55.984 INFO:teuthology.orchestra.run.smithi066.stdout: "root_rsnaps": 6, 2024-04-25T13:29:55.984 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:29:55.984 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_set_tag": 0, 2024-04-25T13:29:55.984 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:29:55.984 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:29:55.984 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:29:55.984 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:29:55.984 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:29:55.984 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_file_inodes": 0, 2024-04-25T13:29:55.985 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:29:55.985 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:29:55.985 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:29:55.985 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:29:55.985 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:29:55.985 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:29:55.985 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps": 0, 2024-04-25T13:29:55.985 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:29:55.986 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_cap_release": 0, 2024-04-25T13:29:55.986 INFO:teuthology.orchestra.run.smithi066.stdout: "process_request_cap_release": 0 2024-04-25T13:29:55.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.986 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_cache": { 2024-04-25T13:29:55.986 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update": 0, 2024-04-25T13:29:55.986 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_update_receipt": 0, 2024-04-25T13:29:55.986 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_try_discover": 0, 2024-04-25T13:29:55.986 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_send_discover": 0, 2024-04-25T13:29:55.986 INFO:teuthology.orchestra.run.smithi066.stdout: "dir_handle_discover": 0, 2024-04-25T13:29:55.987 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays": 0, 2024-04-25T13:29:55.987 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_delayed": 0, 2024-04-25T13:29:55.987 INFO:teuthology.orchestra.run.smithi066.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:29:55.987 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_created": 0, 2024-04-25T13:29:55.987 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_enqueued": 0, 2024-04-25T13:29:55.987 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_reintegrated": 0, 2024-04-25T13:29:55.987 INFO:teuthology.orchestra.run.smithi066.stdout: "strays_migrated": 0, 2024-04-25T13:29:55.987 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_processing": 0, 2024-04-25T13:29:55.987 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:29:55.988 INFO:teuthology.orchestra.run.smithi066.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:29:55.988 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_started": 0, 2024-04-25T13:29:55.988 INFO:teuthology.orchestra.run.smithi066.stdout: "recovery_completed": 0, 2024-04-25T13:29:55.988 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:29:55.988 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:29:55.988 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:29:55.988 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_exportdir": 0, 2024-04-25T13:29:55.988 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_flush": 0, 2024-04-25T13:29:55.988 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:29:55.989 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_fragstats": 0, 2024-04-25T13:29:55.989 INFO:teuthology.orchestra.run.smithi066.stdout: "ireq_inodestats": 0 2024-04-25T13:29:55.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.989 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_log": { 2024-04-25T13:29:55.989 INFO:teuthology.orchestra.run.smithi066.stdout: "evlrg": 0, 2024-04-25T13:29:55.989 INFO:teuthology.orchestra.run.smithi066.stdout: "evadd": 0, 2024-04-25T13:29:55.989 INFO:teuthology.orchestra.run.smithi066.stdout: "evex": 0, 2024-04-25T13:29:55.989 INFO:teuthology.orchestra.run.smithi066.stdout: "evtrm": 0, 2024-04-25T13:29:55.989 INFO:teuthology.orchestra.run.smithi066.stdout: "ev": 100, 2024-04-25T13:29:55.990 INFO:teuthology.orchestra.run.smithi066.stdout: "evexg": 0, 2024-04-25T13:29:55.990 INFO:teuthology.orchestra.run.smithi066.stdout: "evexd": 0, 2024-04-25T13:29:55.990 INFO:teuthology.orchestra.run.smithi066.stdout: "segadd": 0, 2024-04-25T13:29:55.990 INFO:teuthology.orchestra.run.smithi066.stdout: "segex": 0, 2024-04-25T13:29:55.990 INFO:teuthology.orchestra.run.smithi066.stdout: "segtrm": 0, 2024-04-25T13:29:55.990 INFO:teuthology.orchestra.run.smithi066.stdout: "seg": 2, 2024-04-25T13:29:55.990 INFO:teuthology.orchestra.run.smithi066.stdout: "segmjr": 2, 2024-04-25T13:29:55.990 INFO:teuthology.orchestra.run.smithi066.stdout: "segexg": 0, 2024-04-25T13:29:55.991 INFO:teuthology.orchestra.run.smithi066.stdout: "segexd": 0, 2024-04-25T13:29:55.991 INFO:teuthology.orchestra.run.smithi066.stdout: "expos": 4194304, 2024-04-25T13:29:55.991 INFO:teuthology.orchestra.run.smithi066.stdout: "wrpos": 4283934, 2024-04-25T13:29:55.991 INFO:teuthology.orchestra.run.smithi066.stdout: "rdpos": 4283822, 2024-04-25T13:29:55.991 INFO:teuthology.orchestra.run.smithi066.stdout: "jlat": { 2024-04-25T13:29:55.991 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.991 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.991 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.992 INFO:teuthology.orchestra.run.smithi066.stdout: "replayed": 100 2024-04-25T13:29:55.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.992 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_mem": { 2024-04-25T13:29:55.992 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 13, 2024-04-25T13:29:55.992 INFO:teuthology.orchestra.run.smithi066.stdout: "ino+": 13, 2024-04-25T13:29:55.992 INFO:teuthology.orchestra.run.smithi066.stdout: "ino-": 0, 2024-04-25T13:29:55.992 INFO:teuthology.orchestra.run.smithi066.stdout: "dir": 8, 2024-04-25T13:29:55.992 INFO:teuthology.orchestra.run.smithi066.stdout: "dir+": 8, 2024-04-25T13:29:55.992 INFO:teuthology.orchestra.run.smithi066.stdout: "dir-": 0, 2024-04-25T13:29:55.993 INFO:teuthology.orchestra.run.smithi066.stdout: "dn": 12, 2024-04-25T13:29:55.993 INFO:teuthology.orchestra.run.smithi066.stdout: "dn+": 12, 2024-04-25T13:29:55.993 INFO:teuthology.orchestra.run.smithi066.stdout: "dn-": 0, 2024-04-25T13:29:55.993 INFO:teuthology.orchestra.run.smithi066.stdout: "cap": 0, 2024-04-25T13:29:55.993 INFO:teuthology.orchestra.run.smithi066.stdout: "cap+": 0, 2024-04-25T13:29:55.993 INFO:teuthology.orchestra.run.smithi066.stdout: "cap-": 0, 2024-04-25T13:29:55.993 INFO:teuthology.orchestra.run.smithi066.stdout: "rss": 50808, 2024-04-25T13:29:55.993 INFO:teuthology.orchestra.run.smithi066.stdout: "heap": 215416 2024-04-25T13:29:55.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.994 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_server": { 2024-04-25T13:29:55.994 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_client_request": 0, 2024-04-25T13:29:55.994 INFO:teuthology.orchestra.run.smithi066.stdout: "dispatch_server_request": 0, 2024-04-25T13:29:55.994 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_request": 0, 2024-04-25T13:29:55.994 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_client_session": 0, 2024-04-25T13:29:55.994 INFO:teuthology.orchestra.run.smithi066.stdout: "handle_peer_request": 0, 2024-04-25T13:29:55.994 INFO:teuthology.orchestra.run.smithi066.stdout: "req_create_latency": { 2024-04-25T13:29:55.994 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.995 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.995 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.995 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getattr_latency": { 2024-04-25T13:29:55.995 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.995 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.995 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.995 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getfilelock_latency": { 2024-04-25T13:29:55.996 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.996 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.996 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.996 INFO:teuthology.orchestra.run.smithi066.stdout: "req_link_latency": { 2024-04-25T13:29:55.996 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.996 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.996 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.997 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookup_latency": { 2024-04-25T13:29:55.997 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.997 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.997 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.997 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookuphash_latency": { 2024-04-25T13:29:55.997 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.997 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.997 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.998 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupino_latency": { 2024-04-25T13:29:55.998 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.998 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.998 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.998 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupname_latency": { 2024-04-25T13:29:55.999 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.999 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.999 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:55.999 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupparent_latency": { 2024-04-25T13:29:55.999 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:55.999 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:55.999 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:55.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.000 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lookupsnap_latency": { 2024-04-25T13:29:56.000 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.000 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.000 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.000 INFO:teuthology.orchestra.run.smithi066.stdout: "req_lssnap_latency": { 2024-04-25T13:29:56.000 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.000 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.000 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.001 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mkdir_latency": { 2024-04-25T13:29:56.001 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.002 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.002 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.002 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mknod_latency": { 2024-04-25T13:29:56.002 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.002 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.002 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.003 INFO:teuthology.orchestra.run.smithi066.stdout: "req_mksnap_latency": { 2024-04-25T13:29:56.003 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.003 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.003 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.003 INFO:teuthology.orchestra.run.smithi066.stdout: "req_open_latency": { 2024-04-25T13:29:56.003 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.003 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.004 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.004 INFO:teuthology.orchestra.run.smithi066.stdout: "req_readdir_latency": { 2024-04-25T13:29:56.004 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.004 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.004 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.004 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rename_latency": { 2024-04-25T13:29:56.004 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.005 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.005 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.005 INFO:teuthology.orchestra.run.smithi066.stdout: "req_renamesnap_latency": { 2024-04-25T13:29:56.005 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.005 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.005 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.006 INFO:teuthology.orchestra.run.smithi066.stdout: "req_snapdiff_latency": { 2024-04-25T13:29:56.006 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.006 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.006 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.006 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmdir_latency": { 2024-04-25T13:29:56.006 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.006 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.007 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.007 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmsnap_latency": { 2024-04-25T13:29:56.007 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.007 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.007 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.007 INFO:teuthology.orchestra.run.smithi066.stdout: "req_rmxattr_latency": { 2024-04-25T13:29:56.007 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.008 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.008 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.008 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setattr_latency": { 2024-04-25T13:29:56.008 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.008 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.008 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.009 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setdirlayout_latency": { 2024-04-25T13:29:56.009 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.009 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.009 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.009 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setfilelock_latency": { 2024-04-25T13:29:56.009 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.010 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.010 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.010 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setlayout_latency": { 2024-04-25T13:29:56.010 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.010 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.010 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.010 INFO:teuthology.orchestra.run.smithi066.stdout: "req_setxattr_latency": { 2024-04-25T13:29:56.011 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.011 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.011 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.011 INFO:teuthology.orchestra.run.smithi066.stdout: "req_symlink_latency": { 2024-04-25T13:29:56.011 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.011 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.011 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.012 INFO:teuthology.orchestra.run.smithi066.stdout: "req_unlink_latency": { 2024-04-25T13:29:56.012 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.012 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.012 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.012 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:29:56.012 INFO:teuthology.orchestra.run.smithi066.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:29:56.012 INFO:teuthology.orchestra.run.smithi066.stdout: "req_getvxattr_latency": { 2024-04-25T13:29:56.013 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.013 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.013 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.013 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:29:56.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.013 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_sessions": { 2024-04-25T13:29:56.013 INFO:teuthology.orchestra.run.smithi066.stdout: "session_count": 2, 2024-04-25T13:29:56.013 INFO:teuthology.orchestra.run.smithi066.stdout: "session_add": 5, 2024-04-25T13:29:56.013 INFO:teuthology.orchestra.run.smithi066.stdout: "session_remove": 3, 2024-04-25T13:29:56.014 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_open": 4, 2024-04-25T13:29:56.014 INFO:teuthology.orchestra.run.smithi066.stdout: "sessions_stale": 0, 2024-04-25T13:29:56.014 INFO:teuthology.orchestra.run.smithi066.stdout: "total_load": 0, 2024-04-25T13:29:56.014 INFO:teuthology.orchestra.run.smithi066.stdout: "average_load": 0, 2024-04-25T13:29:56.014 INFO:teuthology.orchestra.run.smithi066.stdout: "avg_session_uptime": 2925, 2024-04-25T13:29:56.014 INFO:teuthology.orchestra.run.smithi066.stdout: "mdthresh_evicted": 0 2024-04-25T13:29:56.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.014 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-04-25T13:29:56.015 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:29:56.015 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-04-25T13:29:56.015 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:29:56.015 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:29:56.015 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:29:56.015 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:29:56.015 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:29:56.015 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:29:56.015 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:29:56.016 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:29:56.016 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:29:56.016 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:29:56.016 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:29:56.016 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:29:56.016 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:29:56.016 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-04-25T13:29:56.016 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:29:56.016 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-04-25T13:29:56.017 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:29:56.017 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:29:56.017 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:29:56.017 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:29:56.017 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:29:56.017 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:29:56.017 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:29:56.017 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-04-25T13:29:56.018 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:29:56.018 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:29:56.018 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:29:56.018 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-04-25T13:29:56.018 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-04-25T13:29:56.018 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-04-25T13:29:56.018 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:29:56.018 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:29:56.018 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:29:56.019 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:29:56.019 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 189410, 2024-04-25T13:29:56.019 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 52, 2024-04-25T13:29:56.019 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:29:56.019 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-04-25T13:29:56.019 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-04-25T13:29:56.019 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-04-25T13:29:56.019 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:29:56.019 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-04-25T13:29:56.020 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:29:56.020 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-04-25T13:29:56.020 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 8416, 2024-04-25T13:29:56.020 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 175, 2024-04-25T13:29:56.020 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:29:56.020 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:29:56.020 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-04-25T13:29:56.020 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-04-25T13:29:56.021 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 47746, 2024-04-25T13:29:56.021 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 220, 2024-04-25T13:29:56.021 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-04-25T13:29:56.021 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-04-25T13:29:56.021 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-04-25T13:29:56.021 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-04-25T13:29:56.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.021 INFO:teuthology.orchestra.run.smithi066.stdout: "objecter": { 2024-04-25T13:29:56.021 INFO:teuthology.orchestra.run.smithi066.stdout: "op_active": 0, 2024-04-25T13:29:56.022 INFO:teuthology.orchestra.run.smithi066.stdout: "op_laggy": 0, 2024-04-25T13:29:56.022 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send": 2228, 2024-04-25T13:29:56.022 INFO:teuthology.orchestra.run.smithi066.stdout: "op_send_bytes": 26, 2024-04-25T13:29:56.022 INFO:teuthology.orchestra.run.smithi066.stdout: "op_resend": 4, 2024-04-25T13:29:56.022 INFO:teuthology.orchestra.run.smithi066.stdout: "op_reply": 2228, 2024-04-25T13:29:56.022 INFO:teuthology.orchestra.run.smithi066.stdout: "op_latency": { 2024-04-25T13:29:56.022 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2228, 2024-04-25T13:29:56.022 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.910994076, 2024-04-25T13:29:56.022 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000408884 2024-04-25T13:29:56.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.023 INFO:teuthology.orchestra.run.smithi066.stdout: "op_inflight": 0, 2024-04-25T13:29:56.023 INFO:teuthology.orchestra.run.smithi066.stdout: "oplen_avg": { 2024-04-25T13:29:56.023 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 2228, 2024-04-25T13:29:56.023 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 2229 2024-04-25T13:29:56.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.023 INFO:teuthology.orchestra.run.smithi066.stdout: "op": 2228, 2024-04-25T13:29:56.023 INFO:teuthology.orchestra.run.smithi066.stdout: "op_r": 2228, 2024-04-25T13:29:56.024 INFO:teuthology.orchestra.run.smithi066.stdout: "op_w": 0, 2024-04-25T13:29:56.024 INFO:teuthology.orchestra.run.smithi066.stdout: "op_rmw": 0, 2024-04-25T13:29:56.024 INFO:teuthology.orchestra.run.smithi066.stdout: "op_pg": 0, 2024-04-25T13:29:56.024 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_stat": 1445, 2024-04-25T13:29:56.024 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_create": 0, 2024-04-25T13:29:56.024 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_read": 780, 2024-04-25T13:29:56.024 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_write": 0, 2024-04-25T13:29:56.024 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writefull": 0, 2024-04-25T13:29:56.024 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_writesame": 0, 2024-04-25T13:29:56.025 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_append": 0, 2024-04-25T13:29:56.025 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_zero": 0, 2024-04-25T13:29:56.025 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_truncate": 0, 2024-04-25T13:29:56.025 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_delete": 0, 2024-04-25T13:29:56.025 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_mapext": 0, 2024-04-25T13:29:56.025 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_sparse_read": 0, 2024-04-25T13:29:56.025 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_clonerange": 0, 2024-04-25T13:29:56.025 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_getxattr": 2, 2024-04-25T13:29:56.025 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_setxattr": 0, 2024-04-25T13:29:56.026 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:29:56.026 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_rmxattr": 0, 2024-04-25T13:29:56.026 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:29:56.026 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_call": 0, 2024-04-25T13:29:56.026 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_watch": 0, 2024-04-25T13:29:56.026 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_notify": 0, 2024-04-25T13:29:56.026 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:29:56.026 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls": 0, 2024-04-25T13:29:56.026 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:29:56.027 INFO:teuthology.orchestra.run.smithi066.stdout: "osdop_other": 0, 2024-04-25T13:29:56.027 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_active": 0, 2024-04-25T13:29:56.027 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_send": 0, 2024-04-25T13:29:56.027 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_resend": 0, 2024-04-25T13:29:56.027 INFO:teuthology.orchestra.run.smithi066.stdout: "linger_ping": 0, 2024-04-25T13:29:56.027 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_active": 0, 2024-04-25T13:29:56.027 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_send": 0, 2024-04-25T13:29:56.027 INFO:teuthology.orchestra.run.smithi066.stdout: "poolop_resend": 0, 2024-04-25T13:29:56.027 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_active": 0, 2024-04-25T13:29:56.028 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_send": 0, 2024-04-25T13:29:56.028 INFO:teuthology.orchestra.run.smithi066.stdout: "poolstat_resend": 0, 2024-04-25T13:29:56.028 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_active": 0, 2024-04-25T13:29:56.028 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_send": 0, 2024-04-25T13:29:56.028 INFO:teuthology.orchestra.run.smithi066.stdout: "statfs_resend": 0, 2024-04-25T13:29:56.028 INFO:teuthology.orchestra.run.smithi066.stdout: "command_active": 0, 2024-04-25T13:29:56.028 INFO:teuthology.orchestra.run.smithi066.stdout: "command_send": 0, 2024-04-25T13:29:56.028 INFO:teuthology.orchestra.run.smithi066.stdout: "command_resend": 0, 2024-04-25T13:29:56.028 INFO:teuthology.orchestra.run.smithi066.stdout: "map_epoch": 104, 2024-04-25T13:29:56.029 INFO:teuthology.orchestra.run.smithi066.stdout: "map_full": 0, 2024-04-25T13:29:56.029 INFO:teuthology.orchestra.run.smithi066.stdout: "map_inc": 16, 2024-04-25T13:29:56.029 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_sessions": 5, 2024-04-25T13:29:56.029 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_open": 5, 2024-04-25T13:29:56.029 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_session_close": 0, 2024-04-25T13:29:56.029 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_laggy": 0, 2024-04-25T13:29:56.029 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_wr": 0, 2024-04-25T13:29:56.029 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_rd": 2, 2024-04-25T13:29:56.029 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_del": 0 2024-04-25T13:29:56.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.030 INFO:teuthology.orchestra.run.smithi066.stdout: "oft": { 2024-04-25T13:29:56.030 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_objs": 0, 2024-04-25T13:29:56.030 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:29:56.030 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_updates": 0, 2024-04-25T13:29:56.030 INFO:teuthology.orchestra.run.smithi066.stdout: "omap_total_removes": 0 2024-04-25T13:29:56.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.030 INFO:teuthology.orchestra.run.smithi066.stdout: "osd-slow-ops": { 2024-04-25T13:29:56.031 INFO:teuthology.orchestra.run.smithi066.stdout: "slow_ops_count": 0 2024-04-25T13:29:56.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.031 INFO:teuthology.orchestra.run.smithi066.stdout: "purge_queue": { 2024-04-25T13:29:56.031 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops": 0, 2024-04-25T13:29:56.031 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:29:56.031 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing": 0, 2024-04-25T13:29:56.031 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executing_high_water": 0, 2024-04-25T13:29:56.031 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed_ops": 0, 2024-04-25T13:29:56.032 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_executed": 0, 2024-04-25T13:29:56.032 INFO:teuthology.orchestra.run.smithi066.stdout: "pq_item_in_journal": 0 2024-04-25T13:29:56.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.032 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:29:56.032 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-25T13:29:56.032 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-04-25T13:29:56.032 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-25T13:29:56.032 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 2463, 2024-04-25T13:29:56.033 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 2091883, 2024-04-25T13:29:56.033 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-25T13:29:56.033 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 2463, 2024-04-25T13:29:56.033 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-04-25T13:29:56.033 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-04-25T13:29:56.033 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 2463, 2024-04-25T13:29:56.033 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 2091883, 2024-04-25T13:29:56.033 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-25T13:29:56.033 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.034 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.034 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.034 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:29:56.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.034 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_bytes": { 2024-04-25T13:29:56.034 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-25T13:29:56.034 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-04-25T13:29:56.034 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-25T13:29:56.034 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-04-25T13:29:56.035 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-04-25T13:29:56.035 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-25T13:29:56.035 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-04-25T13:29:56.035 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 2228, 2024-04-25T13:29:56.035 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 89640, 2024-04-25T13:29:56.035 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 2228, 2024-04-25T13:29:56.036 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 89640, 2024-04-25T13:29:56.036 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-25T13:29:56.036 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.036 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.036 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.036 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:29:56.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.036 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-objecter_ops": { 2024-04-25T13:29:56.036 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-25T13:29:56.037 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 1024, 2024-04-25T13:29:56.037 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-25T13:29:56.037 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-04-25T13:29:56.037 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-04-25T13:29:56.037 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-25T13:29:56.037 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-04-25T13:29:56.037 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 2228, 2024-04-25T13:29:56.037 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 2228, 2024-04-25T13:29:56.038 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 2228, 2024-04-25T13:29:56.038 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 2228, 2024-04-25T13:29:56.038 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-25T13:29:56.038 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.038 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.038 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.038 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:29:56.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.039 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:29:56.039 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-25T13:29:56.039 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-04-25T13:29:56.039 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-25T13:29:56.039 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-04-25T13:29:56.039 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-04-25T13:29:56.039 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-25T13:29:56.039 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-04-25T13:29:56.039 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-04-25T13:29:56.040 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-04-25T13:29:56.040 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-04-25T13:29:56.040 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-04-25T13:29:56.040 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-25T13:29:56.040 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.040 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.040 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.040 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:29:56.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:56.041 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-write_buf_throttle-0x55ce466906e0": { 2024-04-25T13:29:56.041 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-25T13:29:56.041 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 3758096384, 2024-04-25T13:29:56.041 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-25T13:29:56.041 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 0, 2024-04-25T13:29:56.041 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 0, 2024-04-25T13:29:56.041 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-25T13:29:56.041 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 0, 2024-04-25T13:29:56.042 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-04-25T13:29:56.042 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-04-25T13:29:56.042 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 0, 2024-04-25T13:29:56.042 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 0, 2024-04-25T13:29:56.042 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-25T13:29:56.042 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:56.042 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:56.042 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:56.043 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:29:56.043 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:29:56.043 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-25T13:29:56.723 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T13:29:56.723 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:29:56.723 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:29:56.724 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T13:29:56.724 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T13:29:56.983 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:29:56.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:56 smithi031 ceph-mon[29457]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:57.013 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:56 smithi066 ceph-mon[33137]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:57.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:56 smithi136 ceph-mon[33038]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:58.738 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-25T13:29:58.738 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:29:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 4, 2024-04-25T13:29:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 4, 2024-04-25T13:29:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2944, 2024-04-25T13:29:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 448, 2024-04-25T13:29:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-04-25T13:29:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2024-04-25T13:29:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.001576231, 2024-04-25T13:29:58.739 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000041589, 2024-04-25T13:29:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.000599845, 2024-04-25T13:29:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:29:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3, 2024-04-25T13:29:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000847132, 2024-04-25T13:29:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000282377 2024-04-25T13:29:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:58.740 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T13:29:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:29:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:29:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 216, 2024-04-25T13:29:58.741 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 205, 2024-04-25T13:29:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1588016, 2024-04-25T13:29:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 125984, 2024-04-25T13:29:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2024-04-25T13:29:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2024-04-25T13:29:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.045913538, 2024-04-25T13:29:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.013675552, 2024-04-25T13:29:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.036053662, 2024-04-25T13:29:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.005523421, 2024-04-25T13:29:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 204, 2024-04-25T13:29:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.011936414, 2024-04-25T13:29:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000058511 2024-04-25T13:29:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:58.743 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1588016, 2024-04-25T13:29:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 125984 2024-04-25T13:29:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:29:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2, 2024-04-25T13:29:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 157, 2024-04-25T13:29:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:29:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 183616, 2024-04-25T13:29:58.744 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-04-25T13:29:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-04-25T13:29:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.011516356, 2024-04-25T13:29:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.007693790, 2024-04-25T13:29:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.001449062, 2024-04-25T13:29:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:29:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:29:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 154, 2024-04-25T13:29:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.013980180, 2024-04-25T13:29:58.745 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000090780 2024-04-25T13:29:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:29:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:29:58.746 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 183616 2024-04-25T13:29:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-04-25T13:29:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2024-04-25T13:29:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-04-25T13:29:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-04-25T13:29:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:29:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-04-25T13:29:58.747 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:29:58.748 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:29:58.748 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:29:58.748 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:29:58.748 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:29:58.748 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:29:58.748 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:29:58.748 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:29:58.748 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:29:58.748 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:29:58.749 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:29:58.749 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:29:58.749 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:29:58.749 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-04-25T13:29:58.749 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:29:58.749 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-04-25T13:29:58.749 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:29:58.749 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:29:58.749 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:29:58.750 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:29:58.750 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:29:58.750 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:29:58.750 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:29:58.750 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-04-25T13:29:58.750 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:29:58.750 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:29:58.750 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:29:58.751 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-04-25T13:29:58.751 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-04-25T13:29:58.751 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-04-25T13:29:58.751 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:29:58.751 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:29:58.751 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:29:58.751 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:29:58.751 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 149024, 2024-04-25T13:29:58.751 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 31, 2024-04-25T13:29:58.752 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:29:58.752 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-04-25T13:29:58.752 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-04-25T13:29:58.752 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-04-25T13:29:58.752 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:29:58.752 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-04-25T13:29:58.752 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:29:58.752 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-04-25T13:29:58.752 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2024-04-25T13:29:58.753 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2024-04-25T13:29:58.753 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:29:58.753 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:29:58.753 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-04-25T13:29:58.753 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-04-25T13:29:58.753 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2024-04-25T13:29:58.754 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2024-04-25T13:29:58.754 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-04-25T13:29:58.754 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-04-25T13:29:58.754 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-04-25T13:29:58.754 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-04-25T13:29:58.754 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:29:58.754 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:29:58.754 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-25T13:29:58.754 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-04-25T13:29:58.755 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-25T13:29:58.755 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 218, 2024-04-25T13:29:58.755 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1557580, 2024-04-25T13:29:58.755 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-25T13:29:58.755 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 218, 2024-04-25T13:29:58.755 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-04-25T13:29:58.755 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-04-25T13:29:58.755 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 218, 2024-04-25T13:29:58.756 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1557580, 2024-04-25T13:29:58.756 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-25T13:29:58.756 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:29:58.756 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:29:58.756 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:29:58.756 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:29:58.756 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:29:58.756 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-25T13:29:58.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:29:58 smithi031 ceph-mon[29457]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:59.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:29:58 smithi066 ceph-mon[33137]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:59.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:29:58 smithi136 ceph-mon[33038]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:59.441 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-25T13:29:59.441 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-25T13:29:59.441 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-25T13:29:59.442 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T13:29:59.442 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T13:29:59.692 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:30:00.877 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:00 smithi066 ceph-mon[33137]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:00.877 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:00 smithi066 ceph-mon[33137]: overall HEALTH_OK 2024-04-25T13:30:00.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:00 smithi031 ceph-mon[29457]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:00.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:00 smithi031 ceph-mon[29457]: overall HEALTH_OK 2024-04-25T13:30:01.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:00 smithi136 ceph-mon[33038]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:01.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:00 smithi136 ceph-mon[33038]: overall HEALTH_OK 2024-04-25T13:30:01.608 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-25T13:30:01.608 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:30:01.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 0, 2024-04-25T13:30:01.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2, 2024-04-25T13:30:01.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:30:01.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 0, 2024-04-25T13:30:01.610 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-04-25T13:30:01.610 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2024-04-25T13:30:01.610 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.001299489, 2024-04-25T13:30:01.610 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:30:01.610 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.000245655, 2024-04-25T13:30:01.611 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:30:01.611 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:01.611 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:30:01.611 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:30:01.611 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:30:01.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:01.612 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:01.612 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:30:01.612 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:30:01.612 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:30:01.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:01.612 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:30:01.612 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:30:01.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:01.612 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:30:01.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 2, 2024-04-25T13:30:01.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 156, 2024-04-25T13:30:01.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:30:01.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 183440, 2024-04-25T13:30:01.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-04-25T13:30:01.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-04-25T13:30:01.613 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.010892883, 2024-04-25T13:30:01.614 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.006959003, 2024-04-25T13:30:01.614 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.001753585, 2024-04-25T13:30:01.614 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:30:01.614 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:01.614 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 153, 2024-04-25T13:30:01.614 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.018312390, 2024-04-25T13:30:01.614 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000119688 2024-04-25T13:30:01.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:01.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:01.615 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:30:01.615 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:30:01.615 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:30:01.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:01.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:30:01.615 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 183440 2024-04-25T13:30:01.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:01.616 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:30:01.616 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 219, 2024-04-25T13:30:01.616 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 206, 2024-04-25T13:30:01.616 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1590688, 2024-04-25T13:30:01.616 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 125776, 2024-04-25T13:30:01.616 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2024-04-25T13:30:01.616 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2024-04-25T13:30:01.616 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.043802402, 2024-04-25T13:30:01.617 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.013206722, 2024-04-25T13:30:01.617 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.033990270, 2024-04-25T13:30:01.617 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.005130094, 2024-04-25T13:30:01.617 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:01.617 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 206, 2024-04-25T13:30:01.617 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.013382604, 2024-04-25T13:30:01.617 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000064964 2024-04-25T13:30:01.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:01.618 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:01.618 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:30:01.618 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:30:01.618 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:30:01.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:01.618 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1590688, 2024-04-25T13:30:01.618 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 125776 2024-04-25T13:30:01.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:01.619 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-04-25T13:30:01.619 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2024-04-25T13:30:01.619 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-04-25T13:30:01.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:01.619 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-04-25T13:30:01.619 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:30:01.620 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-04-25T13:30:01.620 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:30:01.620 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:30:01.620 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:30:01.620 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:30:01.620 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:30:01.620 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:30:01.620 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:30:01.622 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:30:01.622 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:30:01.622 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:30:01.623 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:30:01.623 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:30:01.623 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:30:01.623 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-04-25T13:30:01.623 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:30:01.623 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-04-25T13:30:01.623 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:30:01.623 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:30:01.624 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:30:01.624 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:30:01.624 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:30:01.624 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:30:01.624 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:30:01.624 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-04-25T13:30:01.624 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:30:01.624 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:30:01.625 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:30:01.625 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-04-25T13:30:01.625 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-04-25T13:30:01.625 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-04-25T13:30:01.625 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:30:01.625 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:30:01.625 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:30:01.625 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:30:01.626 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 150008, 2024-04-25T13:30:01.626 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 31, 2024-04-25T13:30:01.626 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:30:01.626 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-04-25T13:30:01.626 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-04-25T13:30:01.626 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-04-25T13:30:01.626 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:30:01.627 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-04-25T13:30:01.627 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:30:01.627 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-04-25T13:30:01.627 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2024-04-25T13:30:01.627 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2024-04-25T13:30:01.627 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:30:01.627 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:30:01.627 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-04-25T13:30:01.628 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-04-25T13:30:01.628 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2024-04-25T13:30:01.628 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2024-04-25T13:30:01.628 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-04-25T13:30:01.628 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-04-25T13:30:01.628 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-04-25T13:30:01.628 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-04-25T13:30:01.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:01.629 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:30:01.629 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-25T13:30:01.629 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-04-25T13:30:01.629 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-25T13:30:01.629 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 217, 2024-04-25T13:30:01.629 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1557450, 2024-04-25T13:30:01.629 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-25T13:30:01.630 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 217, 2024-04-25T13:30:01.630 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-04-25T13:30:01.630 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-04-25T13:30:01.630 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 217, 2024-04-25T13:30:01.630 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1557450, 2024-04-25T13:30:01.631 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-25T13:30:01.631 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:30:01.631 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:30:01.631 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:30:01.631 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:30:01.631 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:30:01.631 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-25T13:30:02.295 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-04-25T13:30:02.295 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-04-25T13:30:02.295 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-04-25T13:30:02.295 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T13:30:02.296 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T13:30:02.545 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:30:02.952 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:02 smithi066 ceph-mon[33137]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:30:03.021 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:02 smithi136 ceph-mon[33038]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:30:03.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:02 smithi031 ceph-mon[29457]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:30:04.403 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-04-25T13:30:04.403 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:30:04.404 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 0, 2024-04-25T13:30:04.404 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2, 2024-04-25T13:30:04.404 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:30:04.404 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 0, 2024-04-25T13:30:04.404 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2024-04-25T13:30:04.404 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2024-04-25T13:30:04.404 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.001272247, 2024-04-25T13:30:04.404 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:30:04.405 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.000299017, 2024-04-25T13:30:04.405 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:30:04.405 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:04.405 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:30:04.405 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:30:04.405 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:30:04.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:04.405 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:04.406 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:30:04.406 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:30:04.406 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:30:04.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:04.406 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:30:04.406 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:30:04.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:04.406 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:30:04.406 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 220, 2024-04-25T13:30:04.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 209, 2024-04-25T13:30:04.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1589104, 2024-04-25T13:30:04.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 128544, 2024-04-25T13:30:04.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2024-04-25T13:30:04.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2024-04-25T13:30:04.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.046312353, 2024-04-25T13:30:04.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.013689310, 2024-04-25T13:30:04.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.036377807, 2024-04-25T13:30:04.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.005590230, 2024-04-25T13:30:04.408 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:04.408 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 208, 2024-04-25T13:30:04.408 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.010979566, 2024-04-25T13:30:04.408 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000052786 2024-04-25T13:30:04.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:04.408 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:04.408 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:30:04.408 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:30:04.408 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:30:04.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:04.409 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1589104, 2024-04-25T13:30:04.409 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 128544 2024-04-25T13:30:04.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:04.409 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:30:04.409 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 6, 2024-04-25T13:30:04.409 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 161, 2024-04-25T13:30:04.409 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 3264, 2024-04-25T13:30:04.410 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 184416, 2024-04-25T13:30:04.410 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 5, 2024-04-25T13:30:04.410 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2024-04-25T13:30:04.410 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.012105011, 2024-04-25T13:30:04.410 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.007745918, 2024-04-25T13:30:04.410 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.002133893, 2024-04-25T13:30:04.410 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:30:04.410 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:04.410 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 159, 2024-04-25T13:30:04.411 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.015826681, 2024-04-25T13:30:04.411 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000099538 2024-04-25T13:30:04.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:04.411 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:04.411 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:30:04.411 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:30:04.411 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:30:04.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:04.411 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T13:30:04.412 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 184416 2024-04-25T13:30:04.412 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:04.412 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2024-04-25T13:30:04.412 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2024-04-25T13:30:04.412 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2024-04-25T13:30:04.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:04.413 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2024-04-25T13:30:04.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:30:04.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2024-04-25T13:30:04.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:30:04.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:30:04.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:30:04.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:30:04.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:30:04.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:30:04.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:30:04.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:30:04.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:30:04.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:30:04.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:30:04.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:30:04.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:30:04.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2024-04-25T13:30:04.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:30:04.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2024-04-25T13:30:04.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:30:04.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:30:04.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:30:04.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:30:04.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:30:04.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:30:04.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:30:04.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2024-04-25T13:30:04.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:30:04.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:30:04.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:30:04.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2024-04-25T13:30:04.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2024-04-25T13:30:04.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2024-04-25T13:30:04.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:30:04.417 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:30:04.417 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:30:04.417 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:30:04.417 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:30:04.417 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 30, 2024-04-25T13:30:04.417 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:30:04.417 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2024-04-25T13:30:04.417 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2024-04-25T13:30:04.417 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2024-04-25T13:30:04.418 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:30:04.418 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2024-04-25T13:30:04.418 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:30:04.418 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2024-04-25T13:30:04.418 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2024-04-25T13:30:04.418 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2024-04-25T13:30:04.418 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:30:04.418 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:30:04.418 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2024-04-25T13:30:04.419 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2024-04-25T13:30:04.419 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2024-04-25T13:30:04.419 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2024-04-25T13:30:04.419 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2024-04-25T13:30:04.419 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2024-04-25T13:30:04.419 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2024-04-25T13:30:04.419 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2024-04-25T13:30:04.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-04-25T13:30:04.420 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:30:04.420 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2024-04-25T13:30:04.420 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2024-04-25T13:30:04.420 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2024-04-25T13:30:04.420 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 222, 2024-04-25T13:30:04.420 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1558100, 2024-04-25T13:30:04.420 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2024-04-25T13:30:04.420 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 222, 2024-04-25T13:30:04.420 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2024-04-25T13:30:04.421 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2024-04-25T13:30:04.421 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 222, 2024-04-25T13:30:04.421 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1558100, 2024-04-25T13:30:04.421 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2024-04-25T13:30:04.421 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2024-04-25T13:30:04.421 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2024-04-25T13:30:04.421 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2024-04-25T13:30:04.421 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:30:04.421 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-04-25T13:30:04.422 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-04-25T13:30:05.026 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:04 smithi066 ceph-mon[33137]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:30:05.054 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T13:30:05.055 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T13:30:05.055 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T13:30:05.055 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T13:30:05.055 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T13:30:05.082 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:04 smithi136 ceph-mon[33038]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:30:05.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:04 smithi031 ceph-mon[29457]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:30:05.306 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:30:07.026 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:06 smithi136 ceph-mon[33038]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:30:07.061 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-25T13:30:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:30:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 0, 2024-04-25T13:30:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2, 2024-04-25T13:30:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:30:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 0, 2024-04-25T13:30:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 4, 2024-04-25T13:30:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 0, 2024-04-25T13:30:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.001102081, 2024-04-25T13:30:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:30:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.000209395, 2024-04-25T13:30:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:30:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:30:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:30:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:30:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 6, 2024-04-25T13:30:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 160, 2024-04-25T13:30:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 3264, 2024-04-25T13:30:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 184240, 2024-04-25T13:30:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 5, 2024-04-25T13:30:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 3, 2024-04-25T13:30:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.011722751, 2024-04-25T13:30:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.007558240, 2024-04-25T13:30:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.002063234, 2024-04-25T13:30:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:30:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 158, 2024-04-25T13:30:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.014577856, 2024-04-25T13:30:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000092264 2024-04-25T13:30:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T13:30:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 184240 2024-04-25T13:30:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:30:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 218, 2024-04-25T13:30:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 207, 2024-04-25T13:30:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1588624, 2024-04-25T13:30:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 126784, 2024-04-25T13:30:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 2, 2024-04-25T13:30:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 1, 2024-04-25T13:30:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.048534154, 2024-04-25T13:30:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.015388707, 2024-04-25T13:30:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.036070187, 2024-04-25T13:30:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.005552220, 2024-04-25T13:30:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 206, 2024-04-25T13:30:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.010346544, 2024-04-25T13:30:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000050225 2024-04-25T13:30:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 1588624, 2024-04-25T13:30:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 126784 2024-04-25T13:30:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-04-25T13:30:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 0, 2024-04-25T13:30:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-04-25T13:30:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-04-25T13:30:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:30:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-04-25T13:30:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:30:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:30:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:30:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:30:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:30:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:30:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:30:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:30:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:30:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:30:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:30:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:30:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:30:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-04-25T13:30:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:30:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-04-25T13:30:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:30:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:30:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:30:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:30:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:30:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:30:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:30:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-04-25T13:30:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:30:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:30:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:30:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-04-25T13:30:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-04-25T13:30:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-04-25T13:30:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:30:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:30:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:30:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:30:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:30:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 30, 2024-04-25T13:30:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:30:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-04-25T13:30:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-04-25T13:30:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-04-25T13:30:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:30:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-04-25T13:30:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:30:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-04-25T13:30:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 0, 2024-04-25T13:30:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 0, 2024-04-25T13:30:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:30:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:30:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-04-25T13:30:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-04-25T13:30:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 928, 2024-04-25T13:30:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 2, 2024-04-25T13:30:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-04-25T13:30:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-04-25T13:30:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-04-25T13:30:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-04-25T13:30:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:30:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T13:30:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T13:30:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T13:30:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 220, 2024-04-25T13:30:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 1557904, 2024-04-25T13:30:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T13:30:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 220, 2024-04-25T13:30:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T13:30:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T13:30:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 220, 2024-04-25T13:30:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1557904, 2024-04-25T13:30:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T13:30:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:07.080 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-25T13:30:07.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:06 smithi066 ceph-mon[33137]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:30:07.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:06 smithi031 ceph-mon[29457]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:30:07.763 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-04-25T13:30:07.763 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-04-25T13:30:07.763 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-04-25T13:30:07.763 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T13:30:07.763 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T13:30:08.055 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:30:08.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:08 smithi136 ceph-mon[33038]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:30:08.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:08 smithi136 ceph-mon[33038]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T13:30:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:08 smithi066 ceph-mon[33137]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:30:09.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:08 smithi066 ceph-mon[33137]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T13:30:09.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:08 smithi031 ceph-mon[29457]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:30:09.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:08 smithi031 ceph-mon[29457]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T13:30:09.839 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-25T13:30:09.839 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:30:09.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 4, 2024-04-25T13:30:09.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 4, 2024-04-25T13:30:09.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 2944, 2024-04-25T13:30:09.839 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 448, 2024-04-25T13:30:09.840 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 4, 2024-04-25T13:30:09.840 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 0, 2024-04-25T13:30:09.840 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.001333572, 2024-04-25T13:30:09.840 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.000030059, 2024-04-25T13:30:09.840 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.000537505, 2024-04-25T13:30:09.840 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:30:09.840 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:09.840 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3, 2024-04-25T13:30:09.841 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000738582, 2024-04-25T13:30:09.841 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000246194 2024-04-25T13:30:09.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:09.841 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:09.841 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:09.841 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:09.841 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:09.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:09.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T13:30:09.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:30:09.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:09.842 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:30:09.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2, 2024-04-25T13:30:09.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 158, 2024-04-25T13:30:09.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:30:09.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 183792, 2024-04-25T13:30:09.842 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 5, 2024-04-25T13:30:09.843 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 3, 2024-04-25T13:30:09.843 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.011512094, 2024-04-25T13:30:09.843 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.007938726, 2024-04-25T13:30:09.843 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.001490419, 2024-04-25T13:30:09.843 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:30:09.843 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:09.843 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 155, 2024-04-25T13:30:09.843 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.014107540, 2024-04-25T13:30:09.844 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000091016 2024-04-25T13:30:09.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:09.844 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:09.844 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:09.844 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:09.844 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:09.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:09.844 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:30:09.844 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 183792 2024-04-25T13:30:09.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:09.845 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:30:09.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 219, 2024-04-25T13:30:09.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 208, 2024-04-25T13:30:09.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1588896, 2024-04-25T13:30:09.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 127424, 2024-04-25T13:30:09.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 2, 2024-04-25T13:30:09.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 1, 2024-04-25T13:30:09.845 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.044432042, 2024-04-25T13:30:09.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.013340227, 2024-04-25T13:30:09.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.034219263, 2024-04-25T13:30:09.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.005232087, 2024-04-25T13:30:09.846 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:09.846 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 207, 2024-04-25T13:30:09.846 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.012474734, 2024-04-25T13:30:09.846 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000060264 2024-04-25T13:30:09.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:09.847 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:09.847 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:09.847 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:09.847 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:09.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:09.847 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 1588896, 2024-04-25T13:30:09.847 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 127424 2024-04-25T13:30:09.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:09.847 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-04-25T13:30:09.848 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 0, 2024-04-25T13:30:09.848 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-04-25T13:30:09.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:09.848 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-04-25T13:30:09.848 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:30:09.848 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-04-25T13:30:09.848 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:30:09.848 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:30:09.849 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:30:09.849 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:30:09.849 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:30:09.849 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:30:09.849 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:30:09.849 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:30:09.849 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:30:09.850 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:30:09.850 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:30:09.850 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:30:09.850 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:30:09.850 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-04-25T13:30:09.850 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:30:09.850 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-04-25T13:30:09.850 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:30:09.851 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:30:09.851 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:30:09.851 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:30:09.851 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:30:09.851 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:30:09.851 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:30:09.851 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-04-25T13:30:09.851 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:30:09.852 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:30:09.852 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:30:09.852 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-04-25T13:30:09.852 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-04-25T13:30:09.852 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-04-25T13:30:09.852 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:30:09.852 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:30:09.852 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:30:09.852 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:30:09.853 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 149017, 2024-04-25T13:30:09.853 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 31, 2024-04-25T13:30:09.853 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:30:09.853 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-04-25T13:30:09.853 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-04-25T13:30:09.853 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-04-25T13:30:09.854 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:30:09.854 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-04-25T13:30:09.854 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:30:09.854 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-04-25T13:30:09.854 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 0, 2024-04-25T13:30:09.854 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 0, 2024-04-25T13:30:09.854 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:30:09.854 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:30:09.855 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-04-25T13:30:09.855 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-04-25T13:30:09.855 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 928, 2024-04-25T13:30:09.855 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 2, 2024-04-25T13:30:09.855 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-04-25T13:30:09.855 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-04-25T13:30:09.855 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-04-25T13:30:09.855 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-04-25T13:30:09.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:09.856 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:30:09.856 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T13:30:09.856 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T13:30:09.856 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T13:30:09.856 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 221, 2024-04-25T13:30:09.856 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 1558034, 2024-04-25T13:30:09.856 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T13:30:09.856 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 221, 2024-04-25T13:30:09.856 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T13:30:09.857 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T13:30:09.857 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 221, 2024-04-25T13:30:09.857 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1558034, 2024-04-25T13:30:09.857 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T13:30:09.857 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:09.857 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:09.857 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:09.857 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:09.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:09.858 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-25T13:30:10.491 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-25T13:30:10.492 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-25T13:30:10.492 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-25T13:30:10.492 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T13:30:10.492 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T13:30:10.731 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:30:11.092 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:10 smithi136 ceph-mon[33038]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:30:11.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:10 smithi066 ceph-mon[33137]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:30:11.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:10 smithi031 ceph-mon[29457]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:30:12.377 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-25T13:30:12.378 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:30:12.378 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1080, 2024-04-25T13:30:12.378 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 503, 2024-04-25T13:30:12.379 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 409612, 2024-04-25T13:30:12.379 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 310062, 2024-04-25T13:30:12.379 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 10, 2024-04-25T13:30:12.379 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-04-25T13:30:12.380 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.121131071, 2024-04-25T13:30:12.380 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.027182077, 2024-04-25T13:30:12.380 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.044200734, 2024-04-25T13:30:12.380 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.045010433, 2024-04-25T13:30:12.381 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:12.381 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 501, 2024-04-25T13:30:12.381 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.315025569, 2024-04-25T13:30:12.381 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000628793 2024-04-25T13:30:12.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.381 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:12.381 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 890, 2024-04-25T13:30:12.382 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000161657, 2024-04-25T13:30:12.382 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000181 2024-04-25T13:30:12.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.382 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 324448, 2024-04-25T13:30:12.382 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 99232 2024-04-25T13:30:12.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.382 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:30:12.382 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 572, 2024-04-25T13:30:12.383 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 476, 2024-04-25T13:30:12.383 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1716725, 2024-04-25T13:30:12.383 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 207679, 2024-04-25T13:30:12.383 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 7, 2024-04-25T13:30:12.383 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-04-25T13:30:12.383 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.076481473, 2024-04-25T13:30:12.383 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.020873348, 2024-04-25T13:30:12.383 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.047553172, 2024-04-25T13:30:12.384 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.016327330, 2024-04-25T13:30:12.384 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:12.384 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 475, 2024-04-25T13:30:12.384 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.512464220, 2024-04-25T13:30:12.384 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001078872 2024-04-25T13:30:12.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.384 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:12.384 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 105, 2024-04-25T13:30:12.384 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000018888, 2024-04-25T13:30:12.385 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000179 2024-04-25T13:30:12.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.385 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 1666240, 2024-04-25T13:30:12.385 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 140384 2024-04-25T13:30:12.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.385 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:30:12.385 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 999, 2024-04-25T13:30:12.385 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 399, 2024-04-25T13:30:12.386 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 398265, 2024-04-25T13:30:12.386 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 581738, 2024-04-25T13:30:12.386 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 15, 2024-04-25T13:30:12.386 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-04-25T13:30:12.386 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.147940757, 2024-04-25T13:30:12.386 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.033062669, 2024-04-25T13:30:12.386 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.051558954, 2024-04-25T13:30:12.386 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.058325035, 2024-04-25T13:30:12.387 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:12.387 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 397, 2024-04-25T13:30:12.387 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.118146388, 2024-04-25T13:30:12.387 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000297597 2024-04-25T13:30:12.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.387 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:12.387 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 985, 2024-04-25T13:30:12.387 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000232242, 2024-04-25T13:30:12.388 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000235 2024-04-25T13:30:12.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.388 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 98432, 2024-04-25T13:30:12.388 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 481280 2024-04-25T13:30:12.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.388 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-04-25T13:30:12.388 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-04-25T13:30:12.390 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-04-25T13:30:12.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.392 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-04-25T13:30:12.392 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-04-25T13:30:12.392 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-04-25T13:30:12.392 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 227, 2024-04-25T13:30:12.392 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.053468055, 2024-04-25T13:30:12.392 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000235542 2024-04-25T13:30:12.393 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:12.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.393 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-04-25T13:30:12.393 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-04-25T13:30:12.393 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-04-25T13:30:12.393 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-04-25T13:30:12.393 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000008245, 2024-04-25T13:30:12.394 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000004122 2024-04-25T13:30:12.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:12.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.394 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-04-25T13:30:12.394 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 166, 2024-04-25T13:30:12.394 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 166, 2024-04-25T13:30:12.396 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-04-25T13:30:12.396 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 166, 2024-04-25T13:30:12.396 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.129914125, 2024-04-25T13:30:12.396 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000782615 2024-04-25T13:30:12.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.396 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-04-25T13:30:12.396 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-04-25T13:30:12.397 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-04-25T13:30:12.397 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-04-25T13:30:12.397 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 12, 2024-04-25T13:30:12.397 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-04-25T13:30:12.397 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-04-25T13:30:12.397 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 25, 2024-04-25T13:30:12.397 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 2, 2024-04-25T13:30:12.397 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 23, 2024-04-25T13:30:12.398 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-04-25T13:30:12.398 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 22, 2024-04-25T13:30:12.398 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-04-25T13:30:12.398 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 2, 2024-04-25T13:30:12.398 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 2, 2024-04-25T13:30:12.398 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 2, 2024-04-25T13:30:12.398 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 209, 2024-04-25T13:30:12.398 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 201, 2024-04-25T13:30:12.399 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-04-25T13:30:12.399 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-04-25T13:30:12.399 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:30:12.399 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-04-25T13:30:12.399 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-04-25T13:30:12.399 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-04-25T13:30:12.399 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-04-25T13:30:12.400 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-04-25T13:30:12.400 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-04-25T13:30:12.400 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-04-25T13:30:12.400 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-04-25T13:30:12.400 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-04-25T13:30:12.400 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:30:12.400 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-04-25T13:30:12.400 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-04-25T13:30:12.401 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-04-25T13:30:12.402 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 6, 2024-04-25T13:30:12.402 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:30:12.402 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-04-25T13:30:12.402 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:30:12.402 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:30:12.403 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:30:12.403 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:30:12.403 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:30:12.403 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-04-25T13:30:12.403 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:30:12.403 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T13:30:12.403 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 5, 2024-04-25T13:30:12.403 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:30:12.404 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:30:12.404 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:30:12.404 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 14, 2024-04-25T13:30:12.404 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 2, 2024-04-25T13:30:12.404 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-04-25T13:30:12.404 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 17 2024-04-25T13:30:12.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.405 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-04-25T13:30:12.405 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-04-25T13:30:12.405 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-04-25T13:30:12.405 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-04-25T13:30:12.405 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-04-25T13:30:12.405 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-04-25T13:30:12.405 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-04-25T13:30:12.405 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-04-25T13:30:12.406 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:30:12.406 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-04-25T13:30:12.406 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-04-25T13:30:12.406 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-04-25T13:30:12.406 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-04-25T13:30:12.406 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-04-25T13:30:12.406 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:30:12.407 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:30:12.407 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-04-25T13:30:12.407 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-04-25T13:30:12.407 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:30:12.407 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:30:12.407 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:30:12.407 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-04-25T13:30:12.407 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-04-25T13:30:12.408 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:30:12.408 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-04-25T13:30:12.408 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-04-25T13:30:12.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.408 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-04-25T13:30:12.408 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-04-25T13:30:12.408 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 109, 2024-04-25T13:30:12.408 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-04-25T13:30:12.409 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-04-25T13:30:12.409 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 109, 2024-04-25T13:30:12.409 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-04-25T13:30:12.409 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-04-25T13:30:12.409 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 2, 2024-04-25T13:30:12.409 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-04-25T13:30:12.409 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-04-25T13:30:12.410 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-04-25T13:30:12.410 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 2, 2024-04-25T13:30:12.410 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-04-25T13:30:12.410 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-04-25T13:30:12.410 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 4194304, 2024-04-25T13:30:12.410 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 4287304, 2024-04-25T13:30:12.410 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-04-25T13:30:12.411 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-04-25T13:30:12.411 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 86, 2024-04-25T13:30:12.411 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.170176442, 2024-04-25T13:30:12.411 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001978795 2024-04-25T13:30:12.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.411 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-04-25T13:30:12.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.411 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-04-25T13:30:12.412 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 23, 2024-04-25T13:30:12.412 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 23, 2024-04-25T13:30:12.412 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 0, 2024-04-25T13:30:12.412 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 18, 2024-04-25T13:30:12.413 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 18, 2024-04-25T13:30:12.413 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 0, 2024-04-25T13:30:12.413 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 25, 2024-04-25T13:30:12.413 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 25, 2024-04-25T13:30:12.413 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 0, 2024-04-25T13:30:12.413 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 2, 2024-04-25T13:30:12.413 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 17, 2024-04-25T13:30:12.413 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 15, 2024-04-25T13:30:12.414 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 53640, 2024-04-25T13:30:12.414 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 231800 2024-04-25T13:30:12.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.414 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-04-25T13:30:12.414 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 190, 2024-04-25T13:30:12.414 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-04-25T13:30:12.414 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 166, 2024-04-25T13:30:12.415 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 32, 2024-04-25T13:30:12.415 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-04-25T13:30:12.415 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-04-25T13:30:12.415 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4, 2024-04-25T13:30:12.415 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.002952459, 2024-04-25T13:30:12.415 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000738114 2024-04-25T13:30:12.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.415 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-04-25T13:30:12.416 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 114, 2024-04-25T13:30:12.416 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.021031415, 2024-04-25T13:30:12.416 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000184486 2024-04-25T13:30:12.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.416 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-04-25T13:30:12.416 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.416 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.416 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.417 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-04-25T13:30:12.417 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.417 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.417 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.417 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-04-25T13:30:12.417 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4, 2024-04-25T13:30:12.418 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.001232134, 2024-04-25T13:30:12.418 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000308033 2024-04-25T13:30:12.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.418 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-04-25T13:30:12.418 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.418 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.418 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.419 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-04-25T13:30:12.419 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.419 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.419 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.419 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-04-25T13:30:12.419 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.420 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.420 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.420 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-04-25T13:30:12.420 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.420 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.420 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.421 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-04-25T13:30:12.421 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.421 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.421 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.421 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-04-25T13:30:12.421 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 11, 2024-04-25T13:30:12.422 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.001522421, 2024-04-25T13:30:12.422 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000138401 2024-04-25T13:30:12.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.423 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-04-25T13:30:12.423 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6, 2024-04-25T13:30:12.423 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.006684503, 2024-04-25T13:30:12.423 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001114083 2024-04-25T13:30:12.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.423 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-04-25T13:30:12.424 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.424 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.424 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.424 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-04-25T13:30:12.424 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 11, 2024-04-25T13:30:12.424 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.054033783, 2024-04-25T13:30:12.425 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.004912162 2024-04-25T13:30:12.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.425 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-04-25T13:30:12.425 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-04-25T13:30:12.425 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000201966, 2024-04-25T13:30:12.425 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000201966 2024-04-25T13:30:12.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.426 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-04-25T13:30:12.426 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.426 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.426 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.426 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-04-25T13:30:12.426 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2, 2024-04-25T13:30:12.426 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.002309577, 2024-04-25T13:30:12.427 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001154788 2024-04-25T13:30:12.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.427 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-04-25T13:30:12.427 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.427 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.427 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.428 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-04-25T13:30:12.428 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.428 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.428 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.428 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-04-25T13:30:12.428 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.428 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.429 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.429 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-04-25T13:30:12.429 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 5, 2024-04-25T13:30:12.429 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.019138905, 2024-04-25T13:30:12.429 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.003827781 2024-04-25T13:30:12.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.429 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-04-25T13:30:12.430 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.430 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.430 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.430 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-04-25T13:30:12.430 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1, 2024-04-25T13:30:12.430 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000847182, 2024-04-25T13:30:12.431 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000847182 2024-04-25T13:30:12.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.431 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-04-25T13:30:12.431 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.431 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.431 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.432 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-04-25T13:30:12.432 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.432 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.432 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.432 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-04-25T13:30:12.432 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.432 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.433 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.433 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-04-25T13:30:12.433 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 7, 2024-04-25T13:30:12.433 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.019959780, 2024-04-25T13:30:12.433 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002851397 2024-04-25T13:30:12.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.434 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-04-25T13:30:12.434 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.434 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.434 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.434 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-04-25T13:30:12.434 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.434 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.435 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.435 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:30:12.435 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:30:12.435 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-04-25T13:30:12.435 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.435 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.435 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.436 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:12.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.436 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-04-25T13:30:12.436 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 2, 2024-04-25T13:30:12.436 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 5, 2024-04-25T13:30:12.436 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 3, 2024-04-25T13:30:12.436 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 2, 2024-04-25T13:30:12.437 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-04-25T13:30:12.437 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 27, 2024-04-25T13:30:12.437 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 13, 2024-04-25T13:30:12.437 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 625, 2024-04-25T13:30:12.437 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-04-25T13:30:12.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.437 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-04-25T13:30:12.438 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:30:12.438 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-04-25T13:30:12.438 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:30:12.438 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:30:12.438 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:30:12.438 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:30:12.438 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:30:12.438 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:30:12.439 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:30:12.439 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:30:12.439 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:30:12.439 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:30:12.439 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:30:12.439 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:30:12.439 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:30:12.440 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-04-25T13:30:12.440 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:30:12.440 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-04-25T13:30:12.440 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:30:12.440 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:30:12.440 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:30:12.440 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:30:12.440 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:30:12.441 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:30:12.441 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:30:12.441 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-04-25T13:30:12.441 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:30:12.441 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:30:12.441 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:30:12.441 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-04-25T13:30:12.441 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-04-25T13:30:12.442 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-04-25T13:30:12.442 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:30:12.442 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:30:12.442 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:30:12.442 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:30:12.442 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 354675, 2024-04-25T13:30:12.442 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 138, 2024-04-25T13:30:12.443 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 792, 2024-04-25T13:30:12.443 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 9, 2024-04-25T13:30:12.443 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-04-25T13:30:12.443 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-04-25T13:30:12.443 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:30:12.443 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-04-25T13:30:12.443 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:30:12.443 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-04-25T13:30:12.444 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 8536, 2024-04-25T13:30:12.444 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 178, 2024-04-25T13:30:12.444 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:30:12.444 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:30:12.444 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-04-25T13:30:12.444 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-04-25T13:30:12.444 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 93898, 2024-04-25T13:30:12.445 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 539, 2024-04-25T13:30:12.445 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-04-25T13:30:12.445 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-04-25T13:30:12.445 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-04-25T13:30:12.445 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-04-25T13:30:12.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.445 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-04-25T13:30:12.445 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-04-25T13:30:12.446 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-04-25T13:30:12.446 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 647, 2024-04-25T13:30:12.446 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 107227, 2024-04-25T13:30:12.446 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 31, 2024-04-25T13:30:12.446 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 647, 2024-04-25T13:30:12.446 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-04-25T13:30:12.446 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 647, 2024-04-25T13:30:12.447 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 17.123622140, 2024-04-25T13:30:12.447 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.026466185 2024-04-25T13:30:12.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.447 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-04-25T13:30:12.447 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-04-25T13:30:12.447 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 647, 2024-04-25T13:30:12.447 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 680 2024-04-25T13:30:12.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.448 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 647, 2024-04-25T13:30:12.448 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 0, 2024-04-25T13:30:12.448 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 647, 2024-04-25T13:30:12.448 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-04-25T13:30:12.448 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-04-25T13:30:12.448 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 0, 2024-04-25T13:30:12.448 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 11, 2024-04-25T13:30:12.449 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 0, 2024-04-25T13:30:12.449 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 86, 2024-04-25T13:30:12.449 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 21, 2024-04-25T13:30:12.449 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-04-25T13:30:12.449 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-04-25T13:30:12.449 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-04-25T13:30:12.449 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-04-25T13:30:12.450 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 506, 2024-04-25T13:30:12.450 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-04-25T13:30:12.450 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-04-25T13:30:12.450 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-04-25T13:30:12.450 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 0, 2024-04-25T13:30:12.450 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 22, 2024-04-25T13:30:12.450 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:30:12.451 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-04-25T13:30:12.451 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:30:12.451 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-04-25T13:30:12.451 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-04-25T13:30:12.451 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-04-25T13:30:12.451 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:30:12.451 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-04-25T13:30:12.452 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:30:12.452 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-04-25T13:30:12.452 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-04-25T13:30:12.452 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-04-25T13:30:12.452 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-04-25T13:30:12.452 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-04-25T13:30:12.452 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-04-25T13:30:12.453 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-04-25T13:30:12.453 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-04-25T13:30:12.453 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-04-25T13:30:12.453 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-04-25T13:30:12.453 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-04-25T13:30:12.453 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-04-25T13:30:12.453 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-04-25T13:30:12.453 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-04-25T13:30:12.454 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-04-25T13:30:12.454 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-04-25T13:30:12.454 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-04-25T13:30:12.454 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 105, 2024-04-25T13:30:12.454 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-04-25T13:30:12.454 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 18, 2024-04-25T13:30:12.454 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 12, 2024-04-25T13:30:12.455 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 12, 2024-04-25T13:30:12.455 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-04-25T13:30:12.455 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-04-25T13:30:12.455 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 30, 2024-04-25T13:30:12.455 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 0, 2024-04-25T13:30:12.455 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 4 2024-04-25T13:30:12.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.455 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-04-25T13:30:12.456 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-04-25T13:30:12.456 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:30:12.456 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-04-25T13:30:12.456 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-04-25T13:30:12.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.456 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-04-25T13:30:12.456 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-04-25T13:30:12.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.457 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-04-25T13:30:12.457 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-04-25T13:30:12.457 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:30:12.457 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-04-25T13:30:12.457 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-04-25T13:30:12.457 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-04-25T13:30:12.457 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-04-25T13:30:12.458 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-04-25T13:30:12.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.458 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:30:12.458 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T13:30:12.458 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T13:30:12.458 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T13:30:12.458 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 2647, 2024-04-25T13:30:12.459 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 2220744, 2024-04-25T13:30:12.459 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T13:30:12.459 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 2647, 2024-04-25T13:30:12.459 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T13:30:12.459 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T13:30:12.459 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2647, 2024-04-25T13:30:12.459 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2220744, 2024-04-25T13:30:12.460 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T13:30:12.460 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.460 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.460 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:12.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.460 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-04-25T13:30:12.460 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T13:30:12.461 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T13:30:12.461 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T13:30:12.461 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T13:30:12.461 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T13:30:12.461 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T13:30:12.461 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T13:30:12.461 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 647, 2024-04-25T13:30:12.461 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 107227, 2024-04-25T13:30:12.462 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 647, 2024-04-25T13:30:12.462 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 107227, 2024-04-25T13:30:12.462 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T13:30:12.462 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.462 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.462 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:12.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.463 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-04-25T13:30:12.463 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T13:30:12.463 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-04-25T13:30:12.463 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T13:30:12.463 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T13:30:12.463 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T13:30:12.464 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T13:30:12.464 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T13:30:12.464 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 647, 2024-04-25T13:30:12.464 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 647, 2024-04-25T13:30:12.464 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 647, 2024-04-25T13:30:12.464 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 647, 2024-04-25T13:30:12.464 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T13:30:12.464 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.465 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.465 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.465 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:12.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.465 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:30:12.465 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T13:30:12.465 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-04-25T13:30:12.465 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T13:30:12.466 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T13:30:12.466 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T13:30:12.466 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T13:30:12.466 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T13:30:12.466 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T13:30:12.466 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T13:30:12.466 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-04-25T13:30:12.467 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-04-25T13:30:12.467 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T13:30:12.467 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.467 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.467 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:12.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:12.467 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x5584349d4500": { 2024-04-25T13:30:12.468 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T13:30:12.468 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-04-25T13:30:12.468 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T13:30:12.468 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 109, 2024-04-25T13:30:12.468 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 93000, 2024-04-25T13:30:12.468 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T13:30:12.468 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 109, 2024-04-25T13:30:12.468 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T13:30:12.469 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T13:30:12.469 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 86, 2024-04-25T13:30:12.469 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 93000, 2024-04-25T13:30:12.469 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T13:30:12.469 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:12.469 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:12.469 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:12.469 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:12.470 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:12.470 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-25T13:30:13.128 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:12 smithi136 ceph-mon[33038]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:30:13.155 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T13:30:13.155 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=11 2024-04-25T13:30:13.155 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=5 2024-04-25T13:30:13.155 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T13:30:13.156 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid e4b023f4-0304-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T13:30:13.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:12 smithi066 ceph-mon[33137]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:30:13.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:12 smithi031 ceph-mon[29457]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:30:13.396 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:30:14.819 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:14 smithi136 ceph-mon[33038]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:30:15.083 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-25T13:30:15.083 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:30:15.083 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 0, 2024-04-25T13:30:15.083 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2, 2024-04-25T13:30:15.084 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:30:15.084 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 0, 2024-04-25T13:30:15.084 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 4, 2024-04-25T13:30:15.084 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 0, 2024-04-25T13:30:15.084 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.001031990, 2024-04-25T13:30:15.084 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:30:15.084 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.000280468, 2024-04-25T13:30:15.084 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:30:15.085 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:15.085 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:15.085 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:15.085 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:15.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:15.085 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:15.085 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:15.085 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:15.085 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:15.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:15.086 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:30:15.086 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:30:15.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:15.086 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:30:15.086 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 228, 2024-04-25T13:30:15.086 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 215, 2024-04-25T13:30:15.086 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 1593200, 2024-04-25T13:30:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 131072, 2024-04-25T13:30:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 2, 2024-04-25T13:30:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 1, 2024-04-25T13:30:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.047813752, 2024-04-25T13:30:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.015113786, 2024-04-25T13:30:15.087 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.036487730, 2024-04-25T13:30:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.005629591, 2024-04-25T13:30:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 215, 2024-04-25T13:30:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.011912282, 2024-04-25T13:30:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000055405 2024-04-25T13:30:15.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:15.089 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 1593200, 2024-04-25T13:30:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 131072 2024-04-25T13:30:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:30:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2, 2024-04-25T13:30:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 162, 2024-04-25T13:30:15.090 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:30:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 184496, 2024-04-25T13:30:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 5, 2024-04-25T13:30:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 3, 2024-04-25T13:30:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.011129946, 2024-04-25T13:30:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.007617457, 2024-04-25T13:30:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.001307940, 2024-04-25T13:30:15.091 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:30:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:30:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 159, 2024-04-25T13:30:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.013782385, 2024-04-25T13:30:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000086681 2024-04-25T13:30:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:30:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:15.092 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:30:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 184496 2024-04-25T13:30:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-04-25T13:30:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 0, 2024-04-25T13:30:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-04-25T13:30:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:15.093 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-04-25T13:30:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:30:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-04-25T13:30:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:30:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:30:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:30:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:30:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:30:15.094 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:30:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:30:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:30:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:30:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:30:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:30:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:30:15.095 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:30:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-04-25T13:30:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:30:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-04-25T13:30:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:30:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:30:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:30:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:30:15.096 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:30:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:30:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:30:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-04-25T13:30:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:30:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:30:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:30:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-04-25T13:30:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-04-25T13:30:15.097 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-04-25T13:30:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:30:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:30:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:30:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:30:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 149017, 2024-04-25T13:30:15.098 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 31, 2024-04-25T13:30:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:30:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-04-25T13:30:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-04-25T13:30:15.099 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-04-25T13:30:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:30:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-04-25T13:30:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:30:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-04-25T13:30:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 0, 2024-04-25T13:30:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 0, 2024-04-25T13:30:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:30:15.100 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:30:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-04-25T13:30:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-04-25T13:30:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 928, 2024-04-25T13:30:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 2, 2024-04-25T13:30:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-04-25T13:30:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-04-25T13:30:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-04-25T13:30:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-04-25T13:30:15.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T13:30:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:30:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T13:30:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T13:30:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T13:30:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 226, 2024-04-25T13:30:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 1558684, 2024-04-25T13:30:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T13:30:15.102 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 226, 2024-04-25T13:30:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T13:30:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T13:30:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 226, 2024-04-25T13:30:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1558684, 2024-04-25T13:30:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T13:30:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T13:30:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T13:30:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T13:30:15.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:15.104 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T13:30:15.104 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-25T13:30:15.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:14 smithi066 ceph-mon[33137]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:30:15.234 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:14 smithi031 ceph-mon[29457]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:30:15.755 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T13:30:15.755 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T13:30:15.755 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T13:30:15.755 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T13:30:15.767 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T13:30:15.768 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:30:15.768 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:30:15.790 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-25T13:30:15.790 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:30:15.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:30:16.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.009+0000 7fbc9c519640 1 -- 172.21.15.31:0/3110959761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc941010a0 msgr2=0x7fbc94101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.009+0000 7fbc9c519640 1 --2- 172.21.15.31:0/3110959761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc941010a0 0x7fbc94101480 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009900 tx=0x7fbc9002a410 comp rx=0 tx=0).stop 2024-04-25T13:30:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.009+0000 7fbc9c519640 1 -- 172.21.15.31:0/3110959761 shutdown_connections 2024-04-25T13:30:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.009+0000 7fbc9c519640 1 --2- 172.21.15.31:0/3110959761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc94106170 0x7fbc9410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.009+0000 7fbc9c519640 1 --2- 172.21.15.31:0/3110959761 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc94101a50 0x7fbc94105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.009+0000 7fbc9c519640 1 --2- 172.21.15.31:0/3110959761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc941010a0 0x7fbc94101480 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.009+0000 7fbc9c519640 1 -- 172.21.15.31:0/3110959761 >> 172.21.15.31:0/3110959761 conn(0x7fbc94077f50 msgr2=0x7fbc94078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.009+0000 7fbc9c519640 1 -- 172.21.15.31:0/3110959761 shutdown_connections 2024-04-25T13:30:16.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.009+0000 7fbc9c519640 1 -- 172.21.15.31:0/3110959761 wait complete. 2024-04-25T13:30:16.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9c519640 1 Processor -- start 2024-04-25T13:30:16.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9c519640 1 -- start start 2024-04-25T13:30:16.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9c519640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc941010a0 0x7fbc941986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9c519640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc94106170 0x7fbc94198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9c519640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc9419cfa0 0x7fbc9419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9c519640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc941110b0 con 0x7fbc9419cfa0 2024-04-25T13:30:16.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9c519640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbc94110f30 con 0x7fbc941010a0 2024-04-25T13:30:16.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9c519640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbc94111230 con 0x7fbc94106170 2024-04-25T13:30:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9aab6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc9419cfa0 0x7fbc9419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9aab6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc9419cfa0 0x7fbc9419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36188/0 (socket says 172.21.15.31:36188) 2024-04-25T13:30:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9aab6640 1 -- 172.21.15.31:0/3175966606 learned_addr learned my addr 172.21.15.31:0/3175966606 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc9a2b5640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc941010a0 0x7fbc941986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.010+0000 7fbc99ab4640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc94106170 0x7fbc94198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.011+0000 7fbc9aab6640 1 -- 172.21.15.31:0/3175966606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc94106170 msgr2=0x7fbc94198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.011+0000 7fbc9aab6640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc94106170 0x7fbc94198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.011+0000 7fbc9aab6640 1 -- 172.21.15.31:0/3175966606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc941010a0 msgr2=0x7fbc941986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.011+0000 7fbc9aab6640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc941010a0 0x7fbc941986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.011+0000 7fbc9aab6640 1 -- 172.21.15.31:0/3175966606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9419da90 con 0x7fbc9419cfa0 2024-04-25T13:30:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.011+0000 7fbc9aab6640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc9419cfa0 0x7fbc9419d450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009fd0 tx=0x7fbc8c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.011+0000 7fbc8b7fe640 1 -- 172.21.15.31:0/3175966606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c017070 con 0x7fbc9419cfa0 2024-04-25T13:30:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.011+0000 7fbc8b7fe640 1 -- 172.21.15.31:0/3175966606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc8c003e20 con 0x7fbc9419cfa0 2024-04-25T13:30:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.011+0000 7fbc9c519640 1 -- 172.21.15.31:0/3175966606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9419dd80 con 0x7fbc9419cfa0 2024-04-25T13:30:16.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.011+0000 7fbc8b7fe640 1 -- 172.21.15.31:0/3175966606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c010460 con 0x7fbc9419cfa0 2024-04-25T13:30:16.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.011+0000 7fbc9c519640 1 -- 172.21.15.31:0/3175966606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc941a5520 con 0x7fbc9419cfa0 2024-04-25T13:30:16.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.012+0000 7fbc9c519640 1 -- 172.21.15.31:0/3175966606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc94068d70 con 0x7fbc9419cfa0 2024-04-25T13:30:16.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.013+0000 7fbc8b7fe640 1 -- 172.21.15.31:0/3175966606 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbc8c003970 con 0x7fbc9419cfa0 2024-04-25T13:30:16.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.014+0000 7fbc8b7fe640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fbc800a3d20 0x7fbc800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.014+0000 7fbc8b7fe640 1 -- 172.21.15.31:0/3175966606 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7fbc8c0d04c0 con 0x7fbc9419cfa0 2024-04-25T13:30:16.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.014+0000 7fbc9a2b5640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fbc800a3d20 0x7fbc800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.014+0000 7fbc9a2b5640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fbc800a3d20 0x7fbc800a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc90031440 tx=0x7fbc90005b50 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:16.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.018+0000 7fbc8b7fe640 1 -- 172.21.15.31:0/3175966606 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc8c09c4a0 con 0x7fbc9419cfa0 2024-04-25T13:30:16.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.208+0000 7fbc9c519640 1 -- 172.21.15.31:0/3175966606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc94199a30 con 0x7fbc9419cfa0 2024-04-25T13:30:16.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.209+0000 7fbc8b7fe640 1 -- 172.21.15.31:0/3175966606 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+23213 (secure 0 0 0) 0x7fbc94199a30 con 0x7fbc9419cfa0 2024-04-25T13:30:16.210 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:30:16.210 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":106,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:30:14.906768+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.2e","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2153915817":"2024-04-26T13:19:35.271090+0000","172.21.15.31:6801/958678691":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/1259986822":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/2022600806":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/958678691":"2024-04-26T13:19:35.271090+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/996243284":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:30:16.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 -- 172.21.15.31:0/3175966606 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fbc800a3d20 msgr2=0x7fbc800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fbc800a3d20 0x7fbc800a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc90031440 tx=0x7fbc90005b50 comp rx=0 tx=0).stop 2024-04-25T13:30:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 -- 172.21.15.31:0/3175966606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc9419cfa0 msgr2=0x7fbc9419d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc9419cfa0 0x7fbc9419d450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009fd0 tx=0x7fbc8c00b260 comp rx=0 tx=0).stop 2024-04-25T13:30:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 -- 172.21.15.31:0/3175966606 shutdown_connections 2024-04-25T13:30:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fbc800a3d20 0x7fbc800a61e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc9419cfa0 0x7fbc9419d450 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc94106170 0x7fbc94198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 --2- 172.21.15.31:0/3175966606 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc941010a0 0x7fbc941986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 -- 172.21.15.31:0/3175966606 >> 172.21.15.31:0/3175966606 conn(0x7fbc94077f50 msgr2=0x7fbc941001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:16.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 -- 172.21.15.31:0/3175966606 shutdown_connections 2024-04-25T13:30:16.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.211+0000 7fbc9c519640 1 -- 172.21.15.31:0/3175966606 wait complete. 2024-04-25T13:30:16.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:15 smithi136 ceph-mon[33038]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T13:30:16.221 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:15 smithi031 ceph-mon[29457]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T13:30:16.223 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:13:01.793580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:17:37.994125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:17:39.116456+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:30:16.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:30:16.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.328+0000 7f2c4e013640 1 -- 172.21.15.31:0/597978973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c48107260 msgr2=0x7f2c4810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.328+0000 7f2c4e013640 1 --2- 172.21.15.31:0/597978973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c48107260 0x7f2c4810ee30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009900 tx=0x7f2c4002a410 comp rx=0 tx=0).stop 2024-04-25T13:30:16.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.328+0000 7f2c4e013640 1 -- 172.21.15.31:0/597978973 shutdown_connections 2024-04-25T13:30:16.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.328+0000 7f2c4e013640 1 --2- 172.21.15.31:0/597978973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c48107260 0x7f2c4810ee30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.328+0000 7f2c4e013640 1 --2- 172.21.15.31:0/597978973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c48102b40 0x7f2c48106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.328+0000 7f2c4e013640 1 --2- 172.21.15.31:0/597978973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c48102190 0x7f2c48102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.328+0000 7f2c4e013640 1 -- 172.21.15.31:0/597978973 >> 172.21.15.31:0/597978973 conn(0x7f2c480793a0 msgr2=0x7f2c480797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:16.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.328+0000 7f2c4e013640 1 -- 172.21.15.31:0/597978973 shutdown_connections 2024-04-25T13:30:16.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.328+0000 7f2c4e013640 1 -- 172.21.15.31:0/597978973 wait complete. 2024-04-25T13:30:16.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.329+0000 7f2c4e013640 1 Processor -- start 2024-04-25T13:30:16.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.329+0000 7f2c4e013640 1 -- start start 2024-04-25T13:30:16.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.329+0000 7f2c4e013640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c48102190 0x7f2c48198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.329+0000 7f2c4e013640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c48107260 0x7f2c48198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.329+0000 7f2c4e013640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c4819cfd0 0x7f2c4819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.329+0000 7f2c4e013640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2c481121b0 con 0x7f2c48107260 2024-04-25T13:30:16.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.329+0000 7f2c4e013640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2c48112030 con 0x7f2c4819cfd0 2024-04-25T13:30:16.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.329+0000 7f2c4e013640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c48112330 con 0x7f2c48102190 2024-04-25T13:30:16.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.329+0000 7f2c46ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c48107260 0x7f2c48198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c46ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c48107260 0x7f2c48198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36216/0 (socket says 172.21.15.31:36216) 2024-04-25T13:30:16.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c46ffd640 1 -- 172.21.15.31:0/55594374 learned_addr learned my addr 172.21.15.31:0/55594374 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:16.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c477fe640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c48102190 0x7f2c48198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c47fff640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c4819cfd0 0x7f2c4819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c46ffd640 1 -- 172.21.15.31:0/55594374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c48102190 msgr2=0x7f2c48198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c46ffd640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c48102190 0x7f2c48198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c46ffd640 1 -- 172.21.15.31:0/55594374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c4819cfd0 msgr2=0x7f2c4819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c46ffd640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c4819cfd0 0x7f2c4819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c46ffd640 1 -- 172.21.15.31:0/55594374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4819dac0 con 0x7f2c48107260 2024-04-25T13:30:16.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c46ffd640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c48107260 0x7f2c48198c50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005fd0 tx=0x7f2c3c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:16.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c44ff9640 1 -- 172.21.15.31:0/55594374 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c017070 con 0x7f2c48107260 2024-04-25T13:30:16.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c44ff9640 1 -- 172.21.15.31:0/55594374 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c3c009c30 con 0x7f2c48107260 2024-04-25T13:30:16.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c4e013640 1 -- 172.21.15.31:0/55594374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4819ddb0 con 0x7f2c48107260 2024-04-25T13:30:16.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c44ff9640 1 -- 172.21.15.31:0/55594374 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c003d30 con 0x7f2c48107260 2024-04-25T13:30:16.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.330+0000 7f2c4e013640 1 -- 172.21.15.31:0/55594374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c481a55a0 con 0x7f2c48107260 2024-04-25T13:30:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.331+0000 7f2c4e013640 1 -- 172.21.15.31:0/55594374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c480691c0 con 0x7f2c48107260 2024-04-25T13:30:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.332+0000 7f2c44ff9640 1 -- 172.21.15.31:0/55594374 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c3c003ed0 con 0x7f2c48107260 2024-04-25T13:30:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.333+0000 7f2c44ff9640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f2c300a3d20 0x7f2c300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.333+0000 7f2c44ff9640 1 -- 172.21.15.31:0/55594374 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7f2c3c0d0cf0 con 0x7f2c48107260 2024-04-25T13:30:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.333+0000 7f2c477fe640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f2c300a3d20 0x7f2c300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.334+0000 7f2c477fe640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f2c300a3d20 0x7f2c300a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2c38005fd0 tx=0x7f2c380098b0 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:16.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.337+0000 7f2c44ff9640 1 -- 172.21.15.31:0/55594374 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c3c020030 con 0x7f2c48107260 2024-04-25T13:30:16.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:15 smithi066 ceph-mon[33137]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T13:30:16.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.525+0000 7f2c4e013640 1 -- 172.21.15.31:0/55594374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2c48102b40 con 0x7f2c48107260 2024-04-25T13:30:16.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.525+0000 7f2c44ff9640 1 -- 172.21.15.31:0/55594374 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7f2c3c09d320 con 0x7f2c48107260 2024-04-25T13:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-25T13:30:16.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 -- 172.21.15.31:0/55594374 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f2c300a3d20 msgr2=0x7f2c300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f2c300a3d20 0x7f2c300a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2c38005fd0 tx=0x7f2c380098b0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 -- 172.21.15.31:0/55594374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c48107260 msgr2=0x7f2c48198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c48107260 0x7f2c48198c50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005fd0 tx=0x7f2c3c00dde0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 -- 172.21.15.31:0/55594374 shutdown_connections 2024-04-25T13:30:16.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f2c300a3d20 0x7f2c300a61e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2c4819cfd0 0x7f2c4819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c48107260 0x7f2c48198c50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 --2- 172.21.15.31:0/55594374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c48102190 0x7f2c48198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 -- 172.21.15.31:0/55594374 >> 172.21.15.31:0/55594374 conn(0x7f2c480793a0 msgr2=0x7f2c480ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:16.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 -- 172.21.15.31:0/55594374 shutdown_connections 2024-04-25T13:30:16.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.527+0000 7f2c4e013640 1 -- 172.21.15.31:0/55594374 wait complete. 2024-04-25T13:30:16.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:30:16.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.643+0000 7fe7e5151640 1 -- 172.21.15.31:0/4042641250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e01010a0 msgr2=0x7fe7e0101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.643+0000 7fe7e5151640 1 --2- 172.21.15.31:0/4042641250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e01010a0 0x7fe7e0101500 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009900 tx=0x7fe7d402a3f0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.643+0000 7fe7e5151640 1 -- 172.21.15.31:0/4042641250 shutdown_connections 2024-04-25T13:30:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.643+0000 7fe7e5151640 1 --2- 172.21.15.31:0/4042641250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e0101a40 0x7fe7e010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.643+0000 7fe7e5151640 1 --2- 172.21.15.31:0/4042641250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e01010a0 0x7fe7e0101500 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.643+0000 7fe7e5151640 1 --2- 172.21.15.31:0/4042641250 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7e0105dd0 0x7fe7e01061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.643+0000 7fe7e5151640 1 -- 172.21.15.31:0/4042641250 >> 172.21.15.31:0/4042641250 conn(0x7fe7e0077c40 msgr2=0x7fe7e0078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.643+0000 7fe7e5151640 1 -- 172.21.15.31:0/4042641250 shutdown_connections 2024-04-25T13:30:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.643+0000 7fe7e5151640 1 -- 172.21.15.31:0/4042641250 wait complete. 2024-04-25T13:30:16.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.643+0000 7fe7e5151640 1 Processor -- start 2024-04-25T13:30:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7e5151640 1 -- start start 2024-04-25T13:30:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7e5151640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01010a0 0x7fe7e0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7e5151640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7e0101a40 0x7fe7e0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7e5151640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0105dd0 0x7fe7e019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7e5151640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e01122d0 con 0x7fe7e0105dd0 2024-04-25T13:30:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7e5151640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0112150 con 0x7fe7e0101a40 2024-04-25T13:30:16.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7e5151640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0112450 con 0x7fe7e01010a0 2024-04-25T13:30:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7df59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0105dd0 0x7fe7e019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7df59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0105dd0 0x7fe7e019d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36244/0 (socket says 172.21.15.31:36244) 2024-04-25T13:30:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7df59e640 1 -- 172.21.15.31:0/2534759337 learned_addr learned my addr 172.21.15.31:0/2534759337 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7de59c640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7e0101a40 0x7fe7e0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7ded9d640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01010a0 0x7fe7e0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7df59e640 1 -- 172.21.15.31:0/2534759337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01010a0 msgr2=0x7fe7e0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7df59e640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01010a0 0x7fe7e0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7df59e640 1 -- 172.21.15.31:0/2534759337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7e0101a40 msgr2=0x7fe7e0198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7df59e640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7e0101a40 0x7fe7e0198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7df59e640 1 -- 172.21.15.31:0/2534759337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e019d740 con 0x7fe7e0105dd0 2024-04-25T13:30:16.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.644+0000 7fe7ded9d640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01010a0 0x7fe7e0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:30:16.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.645+0000 7fe7df59e640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0105dd0 0x7fe7e019d010 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800b970 tx=0x7fe7d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:16.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.645+0000 7fe7c7fff640 1 -- 172.21.15.31:0/2534759337 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8019070 con 0x7fe7e0105dd0 2024-04-25T13:30:16.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.645+0000 7fe7e5151640 1 -- 172.21.15.31:0/2534759337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e019da30 con 0x7fe7e0105dd0 2024-04-25T13:30:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.645+0000 7fe7c7fff640 1 -- 172.21.15.31:0/2534759337 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d8014440 con 0x7fe7e0105dd0 2024-04-25T13:30:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.645+0000 7fe7c7fff640 1 -- 172.21.15.31:0/2534759337 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80075d0 con 0x7fe7e0105dd0 2024-04-25T13:30:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.645+0000 7fe7e5151640 1 -- 172.21.15.31:0/2534759337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e01a52c0 con 0x7fe7e0105dd0 2024-04-25T13:30:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.646+0000 7fe7e5151640 1 -- 172.21.15.31:0/2534759337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e0102540 con 0x7fe7e0105dd0 2024-04-25T13:30:16.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.647+0000 7fe7c7fff640 1 -- 172.21.15.31:0/2534759337 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe7d801c070 con 0x7fe7e0105dd0 2024-04-25T13:30:16.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.647+0000 7fe7c7fff640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe7c80a3d20 0x7fe7c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.648+0000 7fe7c7fff640 1 -- 172.21.15.31:0/2534759337 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7fe7d80d0140 con 0x7fe7e0105dd0 2024-04-25T13:30:16.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.648+0000 7fe7ded9d640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe7c80a3d20 0x7fe7c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.648+0000 7fe7ded9d640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe7c80a3d20 0x7fe7c80a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000abe0 tx=0x7fe7d000a5c0 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:16.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.651+0000 7fe7c7fff640 1 -- 172.21.15.31:0/2534759337 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7d809c2d0 con 0x7fe7e0105dd0 2024-04-25T13:30:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.836+0000 7fe7e5151640 1 -- 172.21.15.31:0/2534759337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7e01a54f0 con 0x7fe7e0105dd0 2024-04-25T13:30:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.836+0000 7fe7c7fff640 1 -- 172.21.15.31:0/2534759337 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7fe7d80a05d0 con 0x7fe7e0105dd0 2024-04-25T13:30:16.837 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-25T13:30:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.838+0000 7fe7e5151640 1 -- 172.21.15.31:0/2534759337 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe7c80a3d20 msgr2=0x7fe7c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.838+0000 7fe7e5151640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe7c80a3d20 0x7fe7c80a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000abe0 tx=0x7fe7d000a5c0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.838+0000 7fe7e5151640 1 -- 172.21.15.31:0/2534759337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0105dd0 msgr2=0x7fe7e019d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.838+0000 7fe7e5151640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0105dd0 0x7fe7e019d010 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800b970 tx=0x7fe7d800be30 comp rx=0 tx=0).stop 2024-04-25T13:30:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.838+0000 7fe7e5151640 1 -- 172.21.15.31:0/2534759337 shutdown_connections 2024-04-25T13:30:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.838+0000 7fe7e5151640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fe7c80a3d20 0x7fe7c80a61e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.838+0000 7fe7e5151640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7e0105dd0 0x7fe7e019d010 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.838+0000 7fe7e5151640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7e0101a40 0x7fe7e0198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.838+0000 7fe7e5151640 1 --2- 172.21.15.31:0/2534759337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e01010a0 0x7fe7e0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.838+0000 7fe7e5151640 1 -- 172.21.15.31:0/2534759337 >> 172.21.15.31:0/2534759337 conn(0x7fe7e0077c40 msgr2=0x7fe7e00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.839+0000 7fe7e5151640 1 -- 172.21.15.31:0/2534759337 shutdown_connections 2024-04-25T13:30:16.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.839+0000 7fe7e5151640 1 -- 172.21.15.31:0/2534759337 wait complete. 2024-04-25T13:30:16.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:30:16.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.955+0000 7f16b3c13640 1 -- 172.21.15.31:0/2329750712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac106170 msgr2=0x7f16ac10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.955+0000 7f16b3c13640 1 --2- 172.21.15.31:0/2329750712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac106170 0x7f16ac10dd40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009ca0 tx=0x7f16a402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.955+0000 7f16b3c13640 1 -- 172.21.15.31:0/2329750712 shutdown_connections 2024-04-25T13:30:16.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.955+0000 7f16b3c13640 1 --2- 172.21.15.31:0/2329750712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac106170 0x7f16ac10dd40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.955+0000 7f16b3c13640 1 --2- 172.21.15.31:0/2329750712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16ac101aa0 0x7f16ac105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.955+0000 7f16b3c13640 1 --2- 172.21.15.31:0/2329750712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16ac1010f0 0x7f16ac1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.955+0000 7f16b3c13640 1 -- 172.21.15.31:0/2329750712 >> 172.21.15.31:0/2329750712 conn(0x7f16ac077fb0 msgr2=0x7f16ac0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:16.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.955+0000 7f16b3c13640 1 -- 172.21.15.31:0/2329750712 shutdown_connections 2024-04-25T13:30:16.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.955+0000 7f16b3c13640 1 -- 172.21.15.31:0/2329750712 wait complete. 2024-04-25T13:30:16.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.955+0000 7f16b3c13640 1 Processor -- start 2024-04-25T13:30:16.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.955+0000 7f16b3c13640 1 -- start start 2024-04-25T13:30:16.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b3c13640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16ac1010f0 0x7f16ac19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b3c13640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16ac106170 0x7f16ac19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b3c13640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac1985f0 0x7f16ac198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b3c13640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac112230 con 0x7f16ac1985f0 2024-04-25T13:30:16.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b3c13640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1120b0 con 0x7f16ac106170 2024-04-25T13:30:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b3c13640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1123b0 con 0x7f16ac1010f0 2024-04-25T13:30:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b21b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac1985f0 0x7f16ac198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b21b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac1985f0 0x7f16ac198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36256/0 (socket says 172.21.15.31:36256) 2024-04-25T13:30:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b21b0640 1 -- 172.21.15.31:0/2283225067 learned_addr learned my addr 172.21.15.31:0/2283225067 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b19af640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16ac1010f0 0x7f16ac19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b11ae640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16ac106170 0x7f16ac19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b21b0640 1 -- 172.21.15.31:0/2283225067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16ac1010f0 msgr2=0x7f16ac19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b21b0640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16ac1010f0 0x7f16ac19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b21b0640 1 -- 172.21.15.31:0/2283225067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16ac106170 msgr2=0x7f16ac19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b21b0640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16ac106170 0x7f16ac19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b21b0640 1 -- 172.21.15.31:0/2283225067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac199110 con 0x7f16ac1985f0 2024-04-25T13:30:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b19af640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16ac1010f0 0x7f16ac19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:30:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b11ae640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16ac106170 0x7f16ac19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:30:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b21b0640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac1985f0 0x7f16ac198aa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f16a40098d0 tx=0x7f16a40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16a2ffd640 1 -- 172.21.15.31:0/2283225067 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4003b70 con 0x7f16ac1985f0 2024-04-25T13:30:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16a2ffd640 1 -- 172.21.15.31:0/2283225067 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a4003d10 con 0x7f16ac1985f0 2024-04-25T13:30:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16b3c13640 1 -- 172.21.15.31:0/2283225067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ac1993a0 con 0x7f16ac1985f0 2024-04-25T13:30:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.956+0000 7f16a2ffd640 1 -- 172.21.15.31:0/2283225067 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a402fa20 con 0x7f16ac1985f0 2024-04-25T13:30:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.957+0000 7f16b3c13640 1 -- 172.21.15.31:0/2283225067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ac1a5640 con 0x7f16ac1985f0 2024-04-25T13:30:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.957+0000 7f16b3c13640 1 -- 172.21.15.31:0/2283225067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ac0684e0 con 0x7f16ac1985f0 2024-04-25T13:30:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.959+0000 7f16a2ffd640 1 -- 172.21.15.31:0/2283225067 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f16a402fbe0 con 0x7f16ac1985f0 2024-04-25T13:30:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.959+0000 7f16a2ffd640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f16980a3da0 0x7f16980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.959+0000 7f16b19af640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f16980a3da0 0x7f16980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.959+0000 7f16a2ffd640 1 -- 172.21.15.31:0/2283225067 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7f16a40f1dd0 con 0x7f16ac1985f0 2024-04-25T13:30:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.960+0000 7f16b19af640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f16980a3da0 0x7f16980a6260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f16a8006fd0 tx=0x7f16a8008040 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:16.963+0000 7f16a2ffd640 1 -- 172.21.15.31:0/2283225067 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16a40be510 con 0x7f16ac1985f0 2024-04-25T13:30:17.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.148+0000 7f16b3c13640 1 -- 172.21.15.31:0/2283225067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f16ac1093a0 con 0x7f16ac1985f0 2024-04-25T13:30:17.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.148+0000 7f16a2ffd640 1 -- 172.21.15.31:0/2283225067 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7f16a40bde10 con 0x7f16ac1985f0 2024-04-25T13:30:17.149 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-04-25T13:30:17.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.150+0000 7f16b3c13640 1 -- 172.21.15.31:0/2283225067 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f16980a3da0 msgr2=0x7f16980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:17.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.150+0000 7f16b3c13640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f16980a3da0 0x7f16980a6260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f16a8006fd0 tx=0x7f16a8008040 comp rx=0 tx=0).stop 2024-04-25T13:30:17.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.150+0000 7f16b3c13640 1 -- 172.21.15.31:0/2283225067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac1985f0 msgr2=0x7f16ac198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:17.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.150+0000 7f16b3c13640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac1985f0 0x7f16ac198aa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f16a40098d0 tx=0x7f16a40317b0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.150+0000 7f16b3c13640 1 -- 172.21.15.31:0/2283225067 shutdown_connections 2024-04-25T13:30:17.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.150+0000 7f16b3c13640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f16980a3da0 0x7f16980a6260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.150+0000 7f16b3c13640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac1985f0 0x7f16ac198aa0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.150+0000 7f16b3c13640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16ac106170 0x7f16ac19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.150+0000 7f16b3c13640 1 --2- 172.21.15.31:0/2283225067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16ac1010f0 0x7f16ac19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.150+0000 7f16b3c13640 1 -- 172.21.15.31:0/2283225067 >> 172.21.15.31:0/2283225067 conn(0x7f16ac077fb0 msgr2=0x7f16ac0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.150+0000 7f16b3c13640 1 -- 172.21.15.31:0/2283225067 shutdown_connections 2024-04-25T13:30:17.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.151+0000 7f16b3c13640 1 -- 172.21.15.31:0/2283225067 wait complete. 2024-04-25T13:30:17.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:16 smithi031 ceph-mon[29457]: pgmap v338: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:30:17.160 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:16 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3175966606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:30:17.161 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:16 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/55594374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:30:17.161 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:16 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2534759337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:30:17.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:30:17.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:16 smithi136 ceph-mon[33038]: pgmap v338: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:30:17.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:16 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3175966606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:30:17.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:16 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/55594374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:30:17.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:16 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2534759337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:30:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.265+0000 7f04f948a640 1 -- 172.21.15.31:0/2122434724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f4101ac0 msgr2=0x7f04f4105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.265+0000 7f04f948a640 1 --2- 172.21.15.31:0/2122434724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f4101ac0 0x7f04f4105ab0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009900 tx=0x7f04e802a410 comp rx=0 tx=0).stop 2024-04-25T13:30:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.265+0000 7f04f948a640 1 -- 172.21.15.31:0/2122434724 shutdown_connections 2024-04-25T13:30:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.265+0000 7f04f948a640 1 --2- 172.21.15.31:0/2122434724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04f4106190 0x7f04f410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.265+0000 7f04f948a640 1 --2- 172.21.15.31:0/2122434724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f4101ac0 0x7f04f4105ab0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.265+0000 7f04f948a640 1 --2- 172.21.15.31:0/2122434724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f4101110 0x7f04f41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.265+0000 7f04f948a640 1 -- 172.21.15.31:0/2122434724 >> 172.21.15.31:0/2122434724 conn(0x7f04f4077fb0 msgr2=0x7f04f40783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.265+0000 7f04f948a640 1 -- 172.21.15.31:0/2122434724 shutdown_connections 2024-04-25T13:30:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.265+0000 7f04f948a640 1 -- 172.21.15.31:0/2122434724 wait complete. 2024-04-25T13:30:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.266+0000 7f04f948a640 1 Processor -- start 2024-04-25T13:30:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.266+0000 7f04f948a640 1 -- start start 2024-04-25T13:30:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.266+0000 7f04f948a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f4101110 0x7f04f4102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.266+0000 7f04f948a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04f4106190 0x7f04f4103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.266+0000 7f04f948a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f41035d0 0x7f04f41a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.266+0000 7f04f948a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41110b0 con 0x7f04f4101110 2024-04-25T13:30:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.266+0000 7f04f948a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4110f30 con 0x7f04f41035d0 2024-04-25T13:30:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.266+0000 7f04f948a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4111230 con 0x7f04f4106190 2024-04-25T13:30:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.266+0000 7f04f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f4101110 0x7f04f4102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.266+0000 7f04f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f4101110 0x7f04f4102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36270/0 (socket says 172.21.15.31:36270) 2024-04-25T13:30:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.266+0000 7f04f2ffd640 1 -- 172.21.15.31:0/1905831426 learned_addr learned my addr 172.21.15.31:0/1905831426 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04f27fc640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04f4106190 0x7f04f4103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04f37fe640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f41035d0 0x7f04f41a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04f2ffd640 1 -- 172.21.15.31:0/1905831426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04f4106190 msgr2=0x7f04f4103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04f2ffd640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04f4106190 0x7f04f4103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04f2ffd640 1 -- 172.21.15.31:0/1905831426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f41035d0 msgr2=0x7f04f41a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04f2ffd640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f41035d0 0x7f04f41a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04f2ffd640 1 -- 172.21.15.31:0/1905831426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f41a6f90 con 0x7f04f4101110 2024-04-25T13:30:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04f37fe640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f41035d0 0x7f04f41a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:30:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04f2ffd640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f4101110 0x7f04f4102b50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f04e400c920 tx=0x7f04e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04dbfff640 1 -- 172.21.15.31:0/1905831426 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4007c70 con 0x7f04f4101110 2024-04-25T13:30:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04f948a640 1 -- 172.21.15.31:0/1905831426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f41a7280 con 0x7f04f4101110 2024-04-25T13:30:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04dbfff640 1 -- 172.21.15.31:0/1905831426 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e4007e10 con 0x7f04f4101110 2024-04-25T13:30:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04dbfff640 1 -- 172.21.15.31:0/1905831426 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4018820 con 0x7f04f4101110 2024-04-25T13:30:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.267+0000 7f04f948a640 1 -- 172.21.15.31:0/1905831426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f41a77a0 con 0x7f04f4101110 2024-04-25T13:30:17.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.268+0000 7f04f948a640 1 -- 172.21.15.31:0/1905831426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f40691c0 con 0x7f04f4101110 2024-04-25T13:30:17.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.269+0000 7f04dbfff640 1 -- 172.21.15.31:0/1905831426 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f04e400f450 con 0x7f04f4101110 2024-04-25T13:30:17.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.270+0000 7f04dbfff640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f04dc0a3d50 0x7f04dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:17.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.270+0000 7f04f27fc640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f04dc0a3d50 0x7f04dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:17.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.270+0000 7f04dbfff640 1 -- 172.21.15.31:0/1905831426 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7f04e40d0a20 con 0x7f04f4101110 2024-04-25T13:30:17.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.271+0000 7f04f27fc640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f04dc0a3d50 0x7f04dc0a6210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f04e8000c00 tx=0x7f04e8032ca0 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:17.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.275+0000 7f04dbfff640 1 -- 172.21.15.31:0/1905831426 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04e4014030 con 0x7f04f4101110 2024-04-25T13:30:17.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:16 smithi066 ceph-mon[33137]: pgmap v338: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:30:17.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:16 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3175966606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:30:17.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:16 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/55594374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:30:17.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:16 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2534759337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:30:17.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.473+0000 7f04f948a640 1 -- 172.21.15.31:0/1905831426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f04f410a3d0 con 0x7f04f4101110 2024-04-25T13:30:17.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.473+0000 7f04dbfff640 1 -- 172.21.15.31:0/1905831426 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v106) ==== 81+0+940 (secure 0 0 0) 0x7f04e401f300 con 0x7f04f4101110 2024-04-25T13:30:17.474 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2153915817 2024-04-26T13:19:35.271090+0000 2024-04-25T13:30:17.474 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/958678691 2024-04-26T13:19:35.271090+0000 2024-04-25T13:30:17.474 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1259986822 2024-04-26T13:19:35.271090+0000 2024-04-25T13:30:17.474 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2022600806 2024-04-26T13:19:35.271090+0000 2024-04-25T13:30:17.474 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1933333469 2024-04-26T13:10:31.702648+0000 2024-04-25T13:30:17.475 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/3047311576 2024-04-26T13:10:31.702648+0000 2024-04-25T13:30:17.475 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/675392433 2024-04-26T13:10:03.802027+0000 2024-04-25T13:30:17.475 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/958678691 2024-04-26T13:19:35.271090+0000 2024-04-25T13:30:17.475 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1458774477 2024-04-26T13:10:03.802027+0000 2024-04-25T13:30:17.475 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/996243284 2024-04-26T13:19:35.271090+0000 2024-04-25T13:30:17.475 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1917834702 2024-04-26T13:10:31.702648+0000 2024-04-25T13:30:17.475 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1458774477 2024-04-26T13:10:03.802027+0000 2024-04-25T13:30:17.476 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3193379806 2024-04-26T13:10:03.802027+0000 2024-04-25T13:30:17.476 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/3047311576 2024-04-26T13:10:31.702648+0000 2024-04-25T13:30:17.476 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/709772735 2024-04-26T13:10:03.802027+0000 2024-04-25T13:30:17.476 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/179341574 2024-04-26T13:10:31.702648+0000 2024-04-25T13:30:17.477 INFO:teuthology.orchestra.run.smithi031.stderr:listed 16 entries 2024-04-25T13:30:17.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 -- 172.21.15.31:0/1905831426 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f04dc0a3d50 msgr2=0x7f04dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:17.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f04dc0a3d50 0x7f04dc0a6210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f04e8000c00 tx=0x7f04e8032ca0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 -- 172.21.15.31:0/1905831426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f4101110 msgr2=0x7f04f4102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:17.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f4101110 0x7f04f4102b50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f04e400c920 tx=0x7f04e400cde0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 -- 172.21.15.31:0/1905831426 shutdown_connections 2024-04-25T13:30:17.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f04dc0a3d50 0x7f04dc0a6210 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f04f41035d0 0x7f04f41a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04f4106190 0x7f04f4103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 --2- 172.21.15.31:0/1905831426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04f4101110 0x7f04f4102b50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 -- 172.21.15.31:0/1905831426 >> 172.21.15.31:0/1905831426 conn(0x7f04f4077fb0 msgr2=0x7f04f406a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:17.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 -- 172.21.15.31:0/1905831426 shutdown_connections 2024-04-25T13:30:17.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.475+0000 7f04f948a640 1 -- 172.21.15.31:0/1905831426 wait complete. 2024-04-25T13:30:17.487 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T13:30:17.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:30:17.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T13:30:17.518 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T13:30:17.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:30:17.518 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:30:17.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:30:17.583 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-25T13:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:30:17.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:30:17.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:30:17.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:30:17.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-04-25T13:30:17.809 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T13:30:17.820 INFO:tasks.cephadm:Teardown begin 2024-04-25T13:30:17.821 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:30:17.821 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:30:17.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.925+0000 7f3dcfccb640 1 -- 172.21.15.31:0/828108195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dc8102240 msgr2=0x7f3dc8102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:17.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.925+0000 7f3dcfccb640 1 --2- 172.21.15.31:0/828108195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dc8102240 0x7f3dc8102620 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009ca0 tx=0x7f3dc002a8e0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.925+0000 7f3dcfccb640 1 -- 172.21.15.31:0/828108195 shutdown_connections 2024-04-25T13:30:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.925+0000 7f3dcfccb640 1 --2- 172.21.15.31:0/828108195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dc8107280 0x7f3dc810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.925+0000 7f3dcfccb640 1 --2- 172.21.15.31:0/828108195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dc8102bf0 0x7f3dc8106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.925+0000 7f3dcfccb640 1 --2- 172.21.15.31:0/828108195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dc8102240 0x7f3dc8102620 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.925+0000 7f3dcfccb640 1 -- 172.21.15.31:0/828108195 >> 172.21.15.31:0/828108195 conn(0x7f3dc80790e0 msgr2=0x7f3dc80794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.925+0000 7f3dcfccb640 1 -- 172.21.15.31:0/828108195 shutdown_connections 2024-04-25T13:30:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.925+0000 7f3dcfccb640 1 -- 172.21.15.31:0/828108195 wait complete. 2024-04-25T13:30:17.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.926+0000 7f3dcfccb640 1 Processor -- start 2024-04-25T13:30:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.926+0000 7f3dcfccb640 1 -- start start 2024-04-25T13:30:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.926+0000 7f3dcfccb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dc8102240 0x7f3dc8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.926+0000 7f3dcfccb640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dc8107280 0x7f3dc8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.926+0000 7f3dcfccb640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dc8074cd0 0x7f3dc81a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.926+0000 7f3dcfccb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8112290 con 0x7f3dc8102240 2024-04-25T13:30:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.926+0000 7f3dcfccb640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8112110 con 0x7f3dc8107280 2024-04-25T13:30:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.926+0000 7f3dcfccb640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8112410 con 0x7f3dc8074cd0 2024-04-25T13:30:17.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.926+0000 7f3dcda67640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dc8102240 0x7f3dc8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:17.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.926+0000 7f3dcda67640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dc8102240 0x7f3dc8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36288/0 (socket says 172.21.15.31:36288) 2024-04-25T13:30:17.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.926+0000 7f3dcda67640 1 -- 172.21.15.31:0/3591565944 learned_addr learned my addr 172.21.15.31:0/3591565944 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:17.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.927+0000 7f3dce268640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dc8074cd0 0x7f3dc81a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:17.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.927+0000 7f3dcd266640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dc8107280 0x7f3dc8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:17.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.927+0000 7f3dcda67640 1 -- 172.21.15.31:0/3591565944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dc8074cd0 msgr2=0x7f3dc81a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:17.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.927+0000 7f3dcda67640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dc8074cd0 0x7f3dc81a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.927+0000 7f3dcda67640 1 -- 172.21.15.31:0/3591565944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dc8107280 msgr2=0x7f3dc8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:17.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.927+0000 7f3dcda67640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dc8107280 0x7f3dc8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:17.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.927+0000 7f3dcda67640 1 -- 172.21.15.31:0/3591565944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc81a6fe0 con 0x7f3dc8102240 2024-04-25T13:30:17.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.927+0000 7f3dcda67640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dc8102240 0x7f3dc8074250 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009dc0 tx=0x7f3dc0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:17.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.929+0000 7f3dc6ffd640 1 -- 172.21.15.31:0/3591565944 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0004240 con 0x7f3dc8102240 2024-04-25T13:30:17.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.929+0000 7f3dc6ffd640 1 -- 172.21.15.31:0/3591565944 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dc0003e40 con 0x7f3dc8102240 2024-04-25T13:30:17.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.929+0000 7f3dcfccb640 1 -- 172.21.15.31:0/3591565944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc81a7270 con 0x7f3dc8102240 2024-04-25T13:30:17.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.929+0000 7f3dc6ffd640 1 -- 172.21.15.31:0/3591565944 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc003a400 con 0x7f3dc8102240 2024-04-25T13:30:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.929+0000 7f3dcfccb640 1 -- 172.21.15.31:0/3591565944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc81a7810 con 0x7f3dc8102240 2024-04-25T13:30:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.930+0000 7f3dcfccb640 1 -- 172.21.15.31:0/3591565944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc80691c0 con 0x7f3dc8102240 2024-04-25T13:30:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.931+0000 7f3dc6ffd640 1 -- 172.21.15.31:0/3591565944 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3dc00043e0 con 0x7f3dc8102240 2024-04-25T13:30:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.931+0000 7f3dc6ffd640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3db00a3da0 0x7f3db00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.932+0000 7f3dc6ffd640 1 -- 172.21.15.31:0/3591565944 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7f3dc00f1b60 con 0x7f3dc8102240 2024-04-25T13:30:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.932+0000 7f3dcd266640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3db00a3da0 0x7f3db00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:17.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.932+0000 7f3dcd266640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3db00a3da0 0x7f3db00a6260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8075c90 tx=0x7f3dbc008040 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:17.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:17.936+0000 7f3dc6ffd640 1 -- 172.21.15.31:0/3591565944 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3dc00098b0 con 0x7f3dc8102240 2024-04-25T13:30:18.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.162+0000 7f3dcfccb640 1 -- 172.21.15.31:0/3591565944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dc810a480 con 0x7f3dc8102240 2024-04-25T13:30:18.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.163+0000 7f3dc6ffd640 1 -- 172.21.15.31:0/3591565944 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11798 (secure 0 0 0) 0x7f3dc00be2a0 con 0x7f3dc8102240 2024-04-25T13:30:18.164 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:30:18.164 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2599371925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2599371925},{"type":"v1","addr":"172.21.15.31:6839","nonce":2599371925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/164386980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":164386980},{"type":"v1","addr":"172.21.15.31:6841","nonce":164386980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:17:40.162672+0000","modified":"2024-04-25T13:17:44.194940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2682191586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2682191586},{"type":"v1","addr":"172.21.15.66:6839","nonce":2682191586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24511]},"id":1}]} 2024-04-25T13:30:18.164 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-04-25T13:30:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.164+0000 7f3dcfccb640 1 -- 172.21.15.31:0/3591565944 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3db00a3da0 msgr2=0x7f3db00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.164+0000 7f3dcfccb640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3db00a3da0 0x7f3db00a6260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8075c90 tx=0x7f3dbc008040 comp rx=0 tx=0).stop 2024-04-25T13:30:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.164+0000 7f3dcfccb640 1 -- 172.21.15.31:0/3591565944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dc8102240 msgr2=0x7f3dc8074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.164+0000 7f3dcfccb640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dc8102240 0x7f3dc8074250 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009dc0 tx=0x7f3dc0003c20 comp rx=0 tx=0).stop 2024-04-25T13:30:18.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.165+0000 7f3dcfccb640 1 -- 172.21.15.31:0/3591565944 shutdown_connections 2024-04-25T13:30:18.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.165+0000 7f3dcfccb640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3db00a3da0 0x7f3db00a6260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.165+0000 7f3dcfccb640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dc8074cd0 0x7f3dc81a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.165+0000 7f3dcfccb640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3dc8107280 0x7f3dc8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.165+0000 7f3dcfccb640 1 --2- 172.21.15.31:0/3591565944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dc8102240 0x7f3dc8074250 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.165+0000 7f3dcfccb640 1 -- 172.21.15.31:0/3591565944 >> 172.21.15.31:0/3591565944 conn(0x7f3dc80790e0 msgr2=0x7f3dc8100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:18.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.165+0000 7f3dcfccb640 1 -- 172.21.15.31:0/3591565944 shutdown_connections 2024-04-25T13:30:18.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.165+0000 7f3dcfccb640 1 -- 172.21.15.31:0/3591565944 wait complete. 2024-04-25T13:30:18.175 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:17 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2283225067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:30:18.175 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:17 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1905831426' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:30:18.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:30:18.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:17 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2283225067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:30:18.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:17 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1905831426' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:30:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.277+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2356508619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388102bf0 msgr2=0x7fd388106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.277+0000 7fd38ecfd640 1 --2- 172.21.15.31:0/2356508619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388102bf0 0x7fd388106ba0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009900 tx=0x7fd37c02a410 comp rx=0 tx=0).stop 2024-04-25T13:30:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.277+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2356508619 shutdown_connections 2024-04-25T13:30:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.277+0000 7fd38ecfd640 1 --2- 172.21.15.31:0/2356508619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd388107280 0x7fd38810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.277+0000 7fd38ecfd640 1 --2- 172.21.15.31:0/2356508619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388102bf0 0x7fd388106ba0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.277+0000 7fd38ecfd640 1 --2- 172.21.15.31:0/2356508619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd388102240 0x7fd388102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.277+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2356508619 >> 172.21.15.31:0/2356508619 conn(0x7fd3880790e0 msgr2=0x7fd3880794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2356508619 shutdown_connections 2024-04-25T13:30:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2356508619 wait complete. 2024-04-25T13:30:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd38ecfd640 1 Processor -- start 2024-04-25T13:30:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd38ecfd640 1 -- start start 2024-04-25T13:30:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd38ecfd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd388102240 0x7fd388198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd38ecfd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388107280 0x7fd388198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd38ecfd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38819d020 0x7fd38819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd38ecfd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881121c0 con 0x7fd388107280 2024-04-25T13:30:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd38ecfd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd388112040 con 0x7fd388102240 2024-04-25T13:30:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd38ecfd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd388112340 con 0x7fd38819d020 2024-04-25T13:30:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd387fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388107280 0x7fd388198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd387fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388107280 0x7fd388198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36306/0 (socket says 172.21.15.31:36306) 2024-04-25T13:30:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd387fff640 1 -- 172.21.15.31:0/2106226121 learned_addr learned my addr 172.21.15.31:0/2106226121 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.278+0000 7fd38ca99640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd388102240 0x7fd388198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd38d29a640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38819d020 0x7fd38819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd387fff640 1 -- 172.21.15.31:0/2106226121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38819d020 msgr2=0x7fd38819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd387fff640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38819d020 0x7fd38819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd387fff640 1 -- 172.21.15.31:0/2106226121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd388102240 msgr2=0x7fd388198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd387fff640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd388102240 0x7fd388198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd387fff640 1 -- 172.21.15.31:0/2106226121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38819db10 con 0x7fd388107280 2024-04-25T13:30:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd38ca99640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd388102240 0x7fd388198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:30:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd387fff640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388107280 0x7fd388198ca0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd37c02a930 tx=0x7fd37c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd385ffb640 1 -- 172.21.15.31:0/2106226121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c03f030 con 0x7fd388107280 2024-04-25T13:30:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd385ffb640 1 -- 172.21.15.31:0/2106226121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd37c004490 con 0x7fd388107280 2024-04-25T13:30:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2106226121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38819dda0 con 0x7fd388107280 2024-04-25T13:30:18.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd385ffb640 1 -- 172.21.15.31:0/2106226121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c035070 con 0x7fd388107280 2024-04-25T13:30:18.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.279+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2106226121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881a55c0 con 0x7fd388107280 2024-04-25T13:30:18.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.280+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2106226121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3880691c0 con 0x7fd388107280 2024-04-25T13:30:18.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.281+0000 7fd385ffb640 1 -- 172.21.15.31:0/2106226121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd37c042030 con 0x7fd388107280 2024-04-25T13:30:18.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.282+0000 7fd385ffb640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd3700a3da0 0x7fd3700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.282+0000 7fd38ca99640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd3700a3da0 0x7fd3700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.282+0000 7fd385ffb640 1 -- 172.21.15.31:0/2106226121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7fd37c0f0ba0 con 0x7fd388107280 2024-04-25T13:30:18.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.283+0000 7fd38ca99640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd3700a3da0 0x7fd3700a6260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd380005fd0 tx=0x7fd380009500 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:18.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.287+0000 7fd385ffb640 1 -- 172.21.15.31:0/2106226121 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd37c0bd170 con 0x7fd388107280 2024-04-25T13:30:18.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:17 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2283225067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:30:18.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:17 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1905831426' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:30:18.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.512+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2106226121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd388102bf0 con 0x7fd388107280 2024-04-25T13:30:18.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.513+0000 7fd385ffb640 1 -- 172.21.15.31:0/2106226121 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11798 (secure 0 0 0) 0x7fd37c0bca70 con 0x7fd388107280 2024-04-25T13:30:18.514 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:30:18.514 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2096804381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":2096804381},{"type":"v1","addr":"172.21.15.31:6835","nonce":2096804381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/1972203542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1972203542},{"type":"v1","addr":"172.21.15.31:6837","nonce":1972203542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2599371925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2599371925},{"type":"v1","addr":"172.21.15.31:6839","nonce":2599371925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/164386980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":164386980},{"type":"v1","addr":"172.21.15.31:6841","nonce":164386980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6833/650781414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":650781414},{"type":"v1","addr":"172.21.15.66:6833","nonce":650781414}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/2316613823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2316613823},{"type":"v1","addr":"172.21.15.136:6833","nonce":2316613823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/2836275175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":2836275175},{"type":"v1","addr":"172.21.15.66:6835","nonce":2836275175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/23034509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":23034509},{"type":"v1","addr":"172.21.15.66:6837","nonce":23034509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3086187130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3086187130},{"type":"v1","addr":"172.21.15.136:6835","nonce":3086187130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3062184830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3062184830},{"type":"v1","addr":"172.21.15.136:6837","nonce":3062184830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:17:40.162672+0000","modified":"2024-04-25T13:17:44.194940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24511},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.66:6839/2682191586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":2682191586},{"type":"v1","addr":"172.21.15.66:6839","nonce":2682191586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6839/2196424437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2196424437},{"type":"v1","addr":"172.21.15.136:6839","nonce":2196424437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24511,"qdb_cluster":[24511]},"id":1}]} 2024-04-25T13:30:18.514 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-04-25T13:30:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2106226121 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd3700a3da0 msgr2=0x7fd3700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd3700a3da0 0x7fd3700a6260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd380005fd0 tx=0x7fd380009500 comp rx=0 tx=0).stop 2024-04-25T13:30:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2106226121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388107280 msgr2=0x7fd388198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388107280 0x7fd388198ca0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd37c02a930 tx=0x7fd37c031a20 comp rx=0 tx=0).stop 2024-04-25T13:30:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2106226121 shutdown_connections 2024-04-25T13:30:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fd3700a3da0 0x7fd3700a6260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38819d020 0x7fd38819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388107280 0x7fd388198ca0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 --2- 172.21.15.31:0/2106226121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd388102240 0x7fd388198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2106226121 >> 172.21.15.31:0/2106226121 conn(0x7fd3880790e0 msgr2=0x7fd3881002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2106226121 shutdown_connections 2024-04-25T13:30:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.515+0000 7fd38ecfd640 1 -- 172.21.15.31:0/2106226121 wait complete. 2024-04-25T13:30:18.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:30:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.627+0000 7f18ac44e640 1 -- 172.21.15.31:0/1826096348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18a4101ac0 msgr2=0x7f18a4105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.627+0000 7f18ac44e640 1 --2- 172.21.15.31:0/1826096348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18a4101ac0 0x7f18a4105ab0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1894009900 tx=0x7f189402a410 comp rx=0 tx=0).stop 2024-04-25T13:30:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.627+0000 7f18ac44e640 1 -- 172.21.15.31:0/1826096348 shutdown_connections 2024-04-25T13:30:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.627+0000 7f18ac44e640 1 --2- 172.21.15.31:0/1826096348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18a4106190 0x7f18a410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.627+0000 7f18ac44e640 1 --2- 172.21.15.31:0/1826096348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18a4101ac0 0x7f18a4105ab0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.627+0000 7f18ac44e640 1 --2- 172.21.15.31:0/1826096348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a4101110 0x7f18a41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.627+0000 7f18ac44e640 1 -- 172.21.15.31:0/1826096348 >> 172.21.15.31:0/1826096348 conn(0x7f18a4077fb0 msgr2=0x7f18a40783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.627+0000 7f18ac44e640 1 -- 172.21.15.31:0/1826096348 shutdown_connections 2024-04-25T13:30:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.627+0000 7f18ac44e640 1 -- 172.21.15.31:0/1826096348 wait complete. 2024-04-25T13:30:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18ac44e640 1 Processor -- start 2024-04-25T13:30:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18ac44e640 1 -- start start 2024-04-25T13:30:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18ac44e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18a4101110 0x7f18a419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18ac44e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18a4106190 0x7f18a419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18ac44e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a41985c0 0x7f18a4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18ac44e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41110d0 con 0x7f18a4101110 2024-04-25T13:30:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18ac44e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f18a4110f50 con 0x7f18a4106190 2024-04-25T13:30:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18ac44e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18a4111250 con 0x7f18a41985c0 2024-04-25T13:30:18.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18aa1ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18a4101110 0x7f18a419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18aa1ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18a4101110 0x7f18a419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36322/0 (socket says 172.21.15.31:36322) 2024-04-25T13:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18aa1ea640 1 -- 172.21.15.31:0/4034054214 learned_addr learned my addr 172.21.15.31:0/4034054214 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18a99e9640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18a4106190 0x7f18a419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18aa9eb640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a41985c0 0x7f18a4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18aa1ea640 1 -- 172.21.15.31:0/4034054214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a41985c0 msgr2=0x7f18a4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18aa1ea640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a41985c0 0x7f18a4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18aa1ea640 1 -- 172.21.15.31:0/4034054214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18a4106190 msgr2=0x7f18a419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18aa1ea640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18a4106190 0x7f18a419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18aa1ea640 1 -- 172.21.15.31:0/4034054214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a41990e0 con 0x7f18a4101110 2024-04-25T13:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.628+0000 7f18aa9eb640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a41985c0 0x7f18a4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.629+0000 7f18aa1ea640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18a4101110 0x7f18a419e410 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f18a000d950 tx=0x7f18a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.629+0000 7f189b7fe640 1 -- 172.21.15.31:0/4034054214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0017070 con 0x7f18a4101110 2024-04-25T13:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.629+0000 7f189b7fe640 1 -- 172.21.15.31:0/4034054214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a0009c30 con 0x7f18a4101110 2024-04-25T13:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.629+0000 7f18ac44e640 1 -- 172.21.15.31:0/4034054214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a41993d0 con 0x7f18a4101110 2024-04-25T13:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.629+0000 7f189b7fe640 1 -- 172.21.15.31:0/4034054214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a000f620 con 0x7f18a4101110 2024-04-25T13:30:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.629+0000 7f18ac44e640 1 -- 172.21.15.31:0/4034054214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a41a5670 con 0x7f18a4101110 2024-04-25T13:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.630+0000 7f18ac44e640 1 -- 172.21.15.31:0/4034054214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a40691c0 con 0x7f18a4101110 2024-04-25T13:30:18.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.631+0000 7f189b7fe640 1 -- 172.21.15.31:0/4034054214 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18a0010460 con 0x7f18a4101110 2024-04-25T13:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.632+0000 7f189b7fe640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f18900a3da0 0x7f18900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.632+0000 7f18a99e9640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f18900a3da0 0x7f18900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.632+0000 7f189b7fe640 1 -- 172.21.15.31:0/4034054214 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7f18a00d0490 con 0x7f18a4101110 2024-04-25T13:30:18.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.632+0000 7f18a99e9640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f18900a3da0 0x7f18900a6260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f1894031400 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:18.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.636+0000 7f189b7fe640 1 -- 172.21.15.31:0/4034054214 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18a009ca60 con 0x7f18a4101110 2024-04-25T13:30:18.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.821+0000 7f18ac44e640 1 -- 172.21.15.31:0/4034054214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18a4199af0 con 0x7f18a4101110 2024-04-25T13:30:18.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.823+0000 7f189b7fe640 1 -- 172.21.15.31:0/4034054214 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+23213 (secure 0 0 0) 0x7f18a0009de0 con 0x7f18a4101110 2024-04-25T13:30:18.825 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-25T13:30:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":106,"fsid":"e4b023f4-0304-11ef-bc93-c7b262605968","created":"2024-04-25T13:09:34.490481+0000","modified":"2024-04-25T13:30:14.906768+0000","last_up_change":"2024-04-25T13:16:52.045599+0000","last_in_change":"2024-04-25T13:16:31.710667+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:13:01.793580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:17:37.994125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:17:39.116456+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5630157c-602a-4837-a0e0-026f57ea83b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6803","nonce":2594221670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6805","nonce":2594221670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6809","nonce":2594221670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2594221670},{"type":"v1","addr":"172.21.15.31:6807","nonce":2594221670}]},"public_addr":"172.21.15.31:6803/2594221670","cluster_addr":"172.21.15.31:6805/2594221670","heartbeat_back_addr":"172.21.15.31:6809/2594221670","heartbeat_front_addr":"172.21.15.31:6807/2594221670","state":["exists","up"]},{"osd":1,"uuid":"f5b3a9a6-6981-4000-a44d-f4ce49325ed6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6801","nonce":786007860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6803","nonce":786007860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6807","nonce":786007860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":786007860},{"type":"v1","addr":"172.21.15.66:6805","nonce":786007860}]},"public_addr":"172.21.15.66:6801/786007860","cluster_addr":"172.21.15.66:6803/786007860","heartbeat_back_addr":"172.21.15.66:6807/786007860","heartbeat_front_addr":"172.21.15.66:6805/786007860","state":["exists","up"]},{"osd":2,"uuid":"d3b167a2-c38c-4b5d-a02c-b155ef73624b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6801","nonce":3445625269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6803","nonce":3445625269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6807","nonce":3445625269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":3445625269},{"type":"v1","addr":"172.21.15.136:6805","nonce":3445625269}]},"public_addr":"172.21.15.136:6801/3445625269","cluster_addr":"172.21.15.136:6803/3445625269","heartbeat_back_addr":"172.21.15.136:6807/3445625269","heartbeat_front_addr":"172.21.15.136:6805/3445625269","state":["exists","up"]},{"osd":3,"uuid":"9dfa360c-0171-4df2-ad03-690209f41d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6811","nonce":1987517958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6813","nonce":1987517958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6817","nonce":1987517958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1987517958},{"type":"v1","addr":"172.21.15.31:6815","nonce":1987517958}]},"public_addr":"172.21.15.31:6811/1987517958","cluster_addr":"172.21.15.31:6813/1987517958","heartbeat_back_addr":"172.21.15.31:6817/1987517958","heartbeat_front_addr":"172.21.15.31:6815/1987517958","state":["exists","up"]},{"osd":4,"uuid":"4f3c0f21-bcc2-4a44-a0e3-3aace3e0d108","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6809","nonce":2164191233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6811","nonce":2164191233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6815","nonce":2164191233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":2164191233},{"type":"v1","addr":"172.21.15.66:6813","nonce":2164191233}]},"public_addr":"172.21.15.66:6809/2164191233","cluster_addr":"172.21.15.66:6811/2164191233","heartbeat_back_addr":"172.21.15.66:6815/2164191233","heartbeat_front_addr":"172.21.15.66:6813/2164191233","state":["exists","up"]},{"osd":5,"uuid":"e70ec561-45b2-4678-9d4a-f94c95a3a29a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6809","nonce":3732026911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6811","nonce":3732026911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6815","nonce":3732026911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3732026911},{"type":"v1","addr":"172.21.15.136:6813","nonce":3732026911}]},"public_addr":"172.21.15.136:6809/3732026911","cluster_addr":"172.21.15.136:6811/3732026911","heartbeat_back_addr":"172.21.15.136:6815/3732026911","heartbeat_front_addr":"172.21.15.136:6813/3732026911","state":["exists","up"]},{"osd":6,"uuid":"00c2e910-8c20-44b2-ab6d-2eb18f8d8add","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6819","nonce":1320917141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6821","nonce":1320917141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6825","nonce":1320917141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1320917141},{"type":"v1","addr":"172.21.15.31:6823","nonce":1320917141}]},"public_addr":"172.21.15.31:6819/1320917141","cluster_addr":"172.21.15.31:6821/1320917141","heartbeat_back_addr":"172.21.15.31:6825/1320917141","heartbeat_front_addr":"172.21.15.31:6823/1320917141","state":["exists","up"]},{"osd":7,"uuid":"50858634-4ead-4b51-9ed1-e5dfe3fd2ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6817","nonce":2705866945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6819","nonce":2705866945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6823","nonce":2705866945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2705866945},{"type":"v1","addr":"172.21.15.66:6821","nonce":2705866945}]},"public_addr":"172.21.15.66:6817/2705866945","cluster_addr":"172.21.15.66:6819/2705866945","heartbeat_back_addr":"172.21.15.66:6823/2705866945","heartbeat_front_addr":"172.21.15.66:6821/2705866945","state":["exists","up"]},{"osd":8,"uuid":"3649fef3-e745-4d64-9601-f09db9203458","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6817","nonce":2823786141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6819","nonce":2823786141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6823","nonce":2823786141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2823786141},{"type":"v1","addr":"172.21.15.136:6821","nonce":2823786141}]},"public_addr":"172.21.15.136:6817/2823786141","cluster_addr":"172.21.15.136:6819/2823786141","heartbeat_back_addr":"172.21.15.136:6823/2823786141","heartbeat_front_addr":"172.21.15.136:6821/2823786141","state":["exists","up"]},{"osd":9,"uuid":"c8974813-c7d4-4aa5-b86f-7c2403d2b29c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6827","nonce":301987340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6829","nonce":301987340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6833","nonce":301987340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":301987340},{"type":"v1","addr":"172.21.15.31:6831","nonce":301987340}]},"public_addr":"172.21.15.31:6827/301987340","cluster_addr":"172.21.15.31:6829/301987340","heartbeat_back_addr":"172.21.15.31:6833/301987340","heartbeat_front_addr":"172.21.15.31:6831/301987340","state":["exists","up"]},{"osd":10,"uuid":"14b500a9-fd99-46fd-8405-c10168928b31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6825","nonce":2563017636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6827","nonce":2563017636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6831","nonce":2563017636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2563017636},{"type":"v1","addr":"172.21.15.66:6829","nonce":2563017636}]},"public_addr":"172.21.15.66:6825/2563017636","cluster_addr":"172.21.15.66:6827/2563017636","heartbeat_back_addr":"172.21.15.66:6831/2563017636","heartbeat_front_addr":"172.21.15.66:6829/2563017636","state":["exists","up"]},{"osd":11,"uuid":"fc059f09-0557-40a5-a573-5c304b9bdc25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6825","nonce":2269044941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6827","nonce":2269044941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6831","nonce":2269044941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2269044941},{"type":"v1","addr":"172.21.15.136:6829","nonce":2269044941}]},"public_addr":"172.21.15.136:6825/2269044941","cluster_addr":"172.21.15.136:6827/2269044941","heartbeat_back_addr":"172.21.15.136:6831/2269044941","heartbeat_front_addr":"172.21.15.136:6829/2269044941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:08.783691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:33.549095+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:12:58.254971+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:24.223948+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:13:49.773802+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:14.790505+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:14:40.699771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:06.767866+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:32.187449+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:15:58.361280+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:23.975809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:16:50.198046+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.2e","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2153915817":"2024-04-26T13:19:35.271090+0000","172.21.15.31:6801/958678691":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/1259986822":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/2022600806":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/1933333469":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6801/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/675392433":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/958678691":"2024-04-26T13:19:35.271090+0000","172.21.15.31:6801/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/996243284":"2024-04-26T13:19:35.271090+0000","172.21.15.31:0/1917834702":"2024-04-26T13:10:31.702648+0000","172.21.15.31:6800/1458774477":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/3193379806":"2024-04-26T13:10:03.802027+0000","172.21.15.31:6800/3047311576":"2024-04-26T13:10:31.702648+0000","172.21.15.31:0/709772735":"2024-04-26T13:10:03.802027+0000","172.21.15.31:0/179341574":"2024-04-26T13:10:31.702648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:30:18.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 -- 172.21.15.31:0/4034054214 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f18900a3da0 msgr2=0x7f18900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f18900a3da0 0x7f18900a6260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f1894031400 comp rx=0 tx=0).stop 2024-04-25T13:30:18.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 -- 172.21.15.31:0/4034054214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18a4101110 msgr2=0x7f18a419e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18a4101110 0x7f18a419e410 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f18a000d950 tx=0x7f18a000de10 comp rx=0 tx=0).stop 2024-04-25T13:30:18.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 -- 172.21.15.31:0/4034054214 shutdown_connections 2024-04-25T13:30:18.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f18900a3da0 0x7f18900a6260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a41985c0 0x7f18a4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18a4106190 0x7f18a419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 --2- 172.21.15.31:0/4034054214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18a4101110 0x7f18a419e410 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 -- 172.21.15.31:0/4034054214 >> 172.21.15.31:0/4034054214 conn(0x7f18a4077fb0 msgr2=0x7f18a406aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:18.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 -- 172.21.15.31:0/4034054214 shutdown_connections 2024-04-25T13:30:18.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.825+0000 7f18ac44e640 1 -- 172.21.15.31:0/4034054214 wait complete. 2024-04-25T13:30:18.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:30:18.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.941+0000 7f3cae0cf640 1 -- 172.21.15.31:0/3986873711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca8101260 msgr2=0x7f3ca810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.941+0000 7f3cae0cf640 1 --2- 172.21.15.31:0/3986873711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca8101260 0x7f3ca810ee30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009cd0 tx=0x7f3ca002a8e0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.941+0000 7f3cae0cf640 1 -- 172.21.15.31:0/3986873711 shutdown_connections 2024-04-25T13:30:18.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.941+0000 7f3cae0cf640 1 --2- 172.21.15.31:0/3986873711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca8101260 0x7f3ca810ee30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.941+0000 7f3cae0cf640 1 --2- 172.21.15.31:0/3986873711 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca81008a0 0x7f3ca8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.941+0000 7f3cae0cf640 1 --2- 172.21.15.31:0/3986873711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca8106f80 0x7f3ca8107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.941+0000 7f3cae0cf640 1 -- 172.21.15.31:0/3986873711 >> 172.21.15.31:0/3986873711 conn(0x7f3ca8076200 msgr2=0x7f3ca8076610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:18.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.941+0000 7f3cae0cf640 1 -- 172.21.15.31:0/3986873711 shutdown_connections 2024-04-25T13:30:18.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.941+0000 7f3cae0cf640 1 -- 172.21.15.31:0/3986873711 wait complete. 2024-04-25T13:30:18.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.941+0000 7f3cae0cf640 1 Processor -- start 2024-04-25T13:30:18.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3cae0cf640 1 -- start start 2024-04-25T13:30:18.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3cae0cf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca81008a0 0x7f3ca819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3cae0cf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca8101260 0x7f3ca819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3cae0cf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca8106f80 0x7f3ca8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3cae0cf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca8112340 con 0x7f3ca8106f80 2024-04-25T13:30:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3cae0cf640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81121c0 con 0x7f3ca81008a0 2024-04-25T13:30:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3cae0cf640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca81124c0 con 0x7f3ca8101260 2024-04-25T13:30:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca8106f80 0x7f3ca8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3ca7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca8106f80 0x7f3ca8198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36342/0 (socket says 172.21.15.31:36342) 2024-04-25T13:30:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3ca7fff640 1 -- 172.21.15.31:0/1373110167 learned_addr learned my addr 172.21.15.31:0/1373110167 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3ca6ffd640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca8101260 0x7f3ca819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3ca77fe640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca81008a0 0x7f3ca819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3ca7fff640 1 -- 172.21.15.31:0/1373110167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca8101260 msgr2=0x7f3ca819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3ca7fff640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca8101260 0x7f3ca819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3ca7fff640 1 -- 172.21.15.31:0/1373110167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca81008a0 msgr2=0x7f3ca819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3ca7fff640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca81008a0 0x7f3ca819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3ca7fff640 1 -- 172.21.15.31:0/1373110167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca8198d80 con 0x7f3ca8106f80 2024-04-25T13:30:18.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.942+0000 7f3ca7fff640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca8106f80 0x7f3ca8198600 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0005cc0 tx=0x7f3ca0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.943+0000 7f3ca4ff9640 1 -- 172.21.15.31:0/1373110167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0004450 con 0x7f3ca8106f80 2024-04-25T13:30:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.943+0000 7f3cae0cf640 1 -- 172.21.15.31:0/1373110167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca8199010 con 0x7f3ca8106f80 2024-04-25T13:30:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.943+0000 7f3ca4ff9640 1 -- 172.21.15.31:0/1373110167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca0031c90 con 0x7f3ca8106f80 2024-04-25T13:30:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.943+0000 7f3ca4ff9640 1 -- 172.21.15.31:0/1373110167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0030e70 con 0x7f3ca8106f80 2024-04-25T13:30:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.943+0000 7f3cae0cf640 1 -- 172.21.15.31:0/1373110167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca81a52c0 con 0x7f3ca8106f80 2024-04-25T13:30:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.944+0000 7f3cae0cf640 1 -- 172.21.15.31:0/1373110167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca80691c0 con 0x7f3ca8106f80 2024-04-25T13:30:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.945+0000 7f3ca4ff9640 1 -- 172.21.15.31:0/1373110167 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3ca00045f0 con 0x7f3ca8106f80 2024-04-25T13:30:18.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.945+0000 7f3ca4ff9640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3c900a3d20 0x7f3c900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.946+0000 7f3ca77fe640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3c900a3d20 0x7f3c900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.946+0000 7f3ca4ff9640 1 -- 172.21.15.31:0/1373110167 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7f3ca002e030 con 0x7f3ca8106f80 2024-04-25T13:30:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.946+0000 7f3ca77fe640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3c900a3d20 0x7f3c900a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009990 tx=0x7f3c98009110 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:18.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:18.950+0000 7f3ca4ff9640 1 -- 172.21.15.31:0/1373110167 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ca004b060 con 0x7f3ca8106f80 2024-04-25T13:30:19.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.176+0000 7f3cae0cf640 1 -- 172.21.15.31:0/1373110167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3ca810a460 con 0x7f3ca8106f80 2024-04-25T13:30:19.179 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:18 smithi031 ceph-mon[29457]: pgmap v339: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 210 B/s wr, 2 op/s 2024-04-25T13:30:19.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:18 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3591565944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:30:19.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:18 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2106226121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:30:19.180 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:18 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/4034054214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:30:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.178+0000 7f3ca4ff9640 1 -- 172.21.15.31:0/1373110167 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7f3ca00098b0 con 0x7f3ca8106f80 2024-04-25T13:30:19.180 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:30:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 -- 172.21.15.31:0/1373110167 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3c900a3d20 msgr2=0x7f3c900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3c900a3d20 0x7f3c900a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009990 tx=0x7f3c98009110 comp rx=0 tx=0).stop 2024-04-25T13:30:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 -- 172.21.15.31:0/1373110167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca8106f80 msgr2=0x7f3ca8198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca8106f80 0x7f3ca8198600 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0005cc0 tx=0x7f3ca0003c20 comp rx=0 tx=0).stop 2024-04-25T13:30:19.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 -- 172.21.15.31:0/1373110167 shutdown_connections 2024-04-25T13:30:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f3c900a3d20 0x7f3c900a61e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca8106f80 0x7f3ca8198600 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca8101260 0x7f3ca819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 --2- 172.21.15.31:0/1373110167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ca81008a0 0x7f3ca819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 -- 172.21.15.31:0/1373110167 >> 172.21.15.31:0/1373110167 conn(0x7f3ca8076200 msgr2=0x7f3ca80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 -- 172.21.15.31:0/1373110167 shutdown_connections 2024-04-25T13:30:19.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.181+0000 7f3cae0cf640 1 -- 172.21.15.31:0/1373110167 wait complete. 2024-04-25T13:30:19.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:30:19.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:18 smithi136 ceph-mon[33038]: pgmap v339: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 210 B/s wr, 2 op/s 2024-04-25T13:30:19.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:18 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3591565944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:30:19.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:18 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2106226121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:30:19.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:18 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/4034054214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:30:19.223 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:18 smithi066 ceph-mon[33137]: pgmap v339: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 210 B/s wr, 2 op/s 2024-04-25T13:30:19.223 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:18 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3591565944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:30:19.223 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:18 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2106226121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:30:19.223 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:18 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/4034054214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:30:19.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.298+0000 7fcc29a91640 1 -- 172.21.15.31:0/347853044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc24101a40 msgr2=0x7fcc24105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:19.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.298+0000 7fcc29a91640 1 --2- 172.21.15.31:0/347853044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc24101a40 0x7fcc24105a80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009900 tx=0x7fcc1802a410 comp rx=0 tx=0).stop 2024-04-25T13:30:19.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.298+0000 7fcc29a91640 1 -- 172.21.15.31:0/347853044 shutdown_connections 2024-04-25T13:30:19.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.298+0000 7fcc29a91640 1 --2- 172.21.15.31:0/347853044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc24106160 0x7fcc2410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:19.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.298+0000 7fcc29a91640 1 --2- 172.21.15.31:0/347853044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc24101a40 0x7fcc24105a80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:19.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.298+0000 7fcc29a91640 1 --2- 172.21.15.31:0/347853044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc24101090 0x7fcc24101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.298+0000 7fcc29a91640 1 -- 172.21.15.31:0/347853044 >> 172.21.15.31:0/347853044 conn(0x7fcc24077f50 msgr2=0x7fcc24078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.298+0000 7fcc29a91640 1 -- 172.21.15.31:0/347853044 shutdown_connections 2024-04-25T13:30:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.298+0000 7fcc29a91640 1 -- 172.21.15.31:0/347853044 wait complete. 2024-04-25T13:30:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc29a91640 1 Processor -- start 2024-04-25T13:30:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc29a91640 1 -- start start 2024-04-25T13:30:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc29a91640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc24101090 0x7fcc24102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc29a91640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc24106160 0x7fcc24103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc29a91640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc241035a0 0x7fcc241a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc29a91640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcc24111080 con 0x7fcc241035a0 2024-04-25T13:30:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc29a91640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcc24110f00 con 0x7fcc24106160 2024-04-25T13:30:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc29a91640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcc24111200 con 0x7fcc24101090 2024-04-25T13:30:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc23fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc241035a0 0x7fcc241a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc23fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc241035a0 0x7fcc241a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36360/0 (socket says 172.21.15.31:36360) 2024-04-25T13:30:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc23fff640 1 -- 172.21.15.31:0/3886233648 learned_addr learned my addr 172.21.15.31:0/3886233648 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc22ffd640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc24106160 0x7fcc24103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:19.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.299+0000 7fcc237fe640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc24101090 0x7fcc24102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:19.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.300+0000 7fcc23fff640 1 -- 172.21.15.31:0/3886233648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc24101090 msgr2=0x7fcc24102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:19.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.300+0000 7fcc23fff640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc24101090 0x7fcc24102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:19.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.300+0000 7fcc23fff640 1 -- 172.21.15.31:0/3886233648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc24106160 msgr2=0x7fcc24103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:19.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.300+0000 7fcc23fff640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc24106160 0x7fcc24103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:19.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.300+0000 7fcc23fff640 1 -- 172.21.15.31:0/3886233648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc241a7020 con 0x7fcc241035a0 2024-04-25T13:30:19.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.300+0000 7fcc23fff640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc241035a0 0x7fcc241a6920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009fd0 tx=0x7fcc1c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:19.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.300+0000 7fcc20ff9640 1 -- 172.21.15.31:0/3886233648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c017070 con 0x7fcc241035a0 2024-04-25T13:30:19.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.300+0000 7fcc20ff9640 1 -- 172.21.15.31:0/3886233648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c003e20 con 0x7fcc241035a0 2024-04-25T13:30:19.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.300+0000 7fcc29a91640 1 -- 172.21.15.31:0/3886233648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc241a7310 con 0x7fcc241035a0 2024-04-25T13:30:19.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.300+0000 7fcc20ff9640 1 -- 172.21.15.31:0/3886233648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c010460 con 0x7fcc241035a0 2024-04-25T13:30:19.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.300+0000 7fcc29a91640 1 -- 172.21.15.31:0/3886233648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc241a77e0 con 0x7fcc241035a0 2024-04-25T13:30:19.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.301+0000 7fcc29a91640 1 -- 172.21.15.31:0/3886233648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc24068d70 con 0x7fcc241035a0 2024-04-25T13:30:19.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.302+0000 7fcc20ff9640 1 -- 172.21.15.31:0/3886233648 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcc1c003970 con 0x7fcc241035a0 2024-04-25T13:30:19.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.303+0000 7fcc20ff9640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fcc0c0a3d50 0x7fcc0c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:19.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.303+0000 7fcc237fe640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fcc0c0a3d50 0x7fcc0c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:19.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.303+0000 7fcc20ff9640 1 -- 172.21.15.31:0/3886233648 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8671+0+0 (secure 0 0 0) 0x7fcc1c0d0540 con 0x7fcc241035a0 2024-04-25T13:30:19.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.303+0000 7fcc237fe640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fcc0c0a3d50 0x7fcc0c0a6210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcc14005fd0 tx=0x7fcc1400b590 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:19.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.307+0000 7fcc20ff9640 1 -- 172.21.15.31:0/3886233648 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc1c09cb50 con 0x7fcc241035a0 2024-04-25T13:30:19.472 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:19 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a[29453]: 2024-04-25T13:30:19.177+0000 7f59f8681700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:30:19.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:19.536+0000 7fcc29a91640 1 -- 172.21.15.31:0/3886233648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcc2410a3a0 con 0x7fcc241035a0 2024-04-25T13:30:19.658 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:30:19 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-b[63639]: ignoring --setuser ceph since I am not root 2024-04-25T13:30:19.659 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:30:19 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-b[63639]: ignoring --setgroup ceph since I am not root 2024-04-25T13:30:19.659 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:30:19 smithi066 ceph-mds[63643]: -- 172.21.15.66:0/2158921557 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a56e11a40 con 0x557a56de4c00 2024-04-25T13:30:19.659 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:30:19 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-b[63639]: starting mds.b at 2024-04-25T13:30:19.719 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:30:19 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i[63643]: ignoring --setuser ceph since I am not root 2024-04-25T13:30:19.719 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:30:19 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i[63643]: ignoring --setgroup ceph since I am not root 2024-04-25T13:30:19.720 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:30:19 smithi136 ceph-mds[63647]: -- 172.21.15.136:0/3571797049 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d49fd1a40 con 0x561d49fa5000 2024-04-25T13:30:19.720 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:30:19 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i[63643]: starting mds.i at 2024-04-25T13:30:20.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.184+0000 7fcc20ff9640 1 -- 172.21.15.31:0/3886233648 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7fcc1c09c450 con 0x7fcc241035a0 2024-04-25T13:30:20.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 -- 172.21.15.31:0/3886233648 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fcc0c0a3d50 msgr2=0x7fcc0c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fcc0c0a3d50 0x7fcc0c0a6210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcc14005fd0 tx=0x7fcc1400b590 comp rx=0 tx=0).stop 2024-04-25T13:30:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 -- 172.21.15.31:0/3886233648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc241035a0 msgr2=0x7fcc241a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc241035a0 0x7fcc241a6920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009fd0 tx=0x7fcc1c00b260 comp rx=0 tx=0).stop 2024-04-25T13:30:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 -- 172.21.15.31:0/3886233648 shutdown_connections 2024-04-25T13:30:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7fcc0c0a3d50 0x7fcc0c0a6210 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc241035a0 0x7fcc241a6920 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:20.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc24106160 0x7fcc24103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:20.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 --2- 172.21.15.31:0/3886233648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc24101090 0x7fcc24102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:20.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 -- 172.21.15.31:0/3886233648 >> 172.21.15.31:0/3886233648 conn(0x7fcc24077f50 msgr2=0x7fcc24069510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:20.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 -- 172.21.15.31:0/3886233648 shutdown_connections 2024-04-25T13:30:20.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.187+0000 7fcc29a91640 1 -- 172.21.15.31:0/3886233648 wait complete. 2024-04-25T13:30:20.199 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:19 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1373110167' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:30:20.199 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:19 smithi031 ceph-mon[29457]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:30:20.199 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:19 smithi031 ceph-mon[29457]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:30:20.200 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:19 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/1373110167' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:30:20.200 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:19 smithi031 ceph-mon[29457]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T13:30:20.200 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:19 smithi031 ceph-mon[29457]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T13:30:20.200 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:19 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3886233648' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:30:20.201 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:30:20.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:19 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1373110167' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:30:20.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:19 smithi136 ceph-mon[33038]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:30:20.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:19 smithi136 ceph-mon[33038]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:30:20.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:19 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/1373110167' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:30:20.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:19 smithi136 ceph-mon[33038]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T13:30:20.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:19 smithi136 ceph-mon[33038]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T13:30:20.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:19 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3886233648' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:30:20.240 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:19 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1373110167' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:30:20.240 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:19 smithi066 ceph-mon[33137]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:30:20.240 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:19 smithi066 ceph-mon[33137]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:30:20.240 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:19 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/1373110167' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:30:20.240 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:19 smithi066 ceph-mon[33137]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T13:30:20.240 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:19 smithi066 ceph-mon[33137]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T13:30:20.241 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:19 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3886233648' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:30:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.308+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/3830207040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c48103af0 msgr2=0x7f0c4810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.308+0000 7f0c4d2ae640 1 --2- 172.21.15.31:0/3830207040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c48103af0 0x7f0c4810bfd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009900 tx=0x7f0c3002a510 comp rx=0 tx=0).stop 2024-04-25T13:30:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.308+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/3830207040 shutdown_connections 2024-04-25T13:30:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.308+0000 7f0c4d2ae640 1 --2- 172.21.15.31:0/3830207040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c4810c510 0x7f0c4810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.308+0000 7f0c4d2ae640 1 --2- 172.21.15.31:0/3830207040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c48103af0 0x7f0c4810bfd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.308+0000 7f0c4d2ae640 1 --2- 172.21.15.31:0/3830207040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c481031d0 0x7f0c481035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:20.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.308+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/3830207040 >> 172.21.15.31:0/3830207040 conn(0x7f0c48077f50 msgr2=0x7f0c48078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:20.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.308+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/3830207040 shutdown_connections 2024-04-25T13:30:20.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.308+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/3830207040 wait complete. 2024-04-25T13:30:20.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.308+0000 7f0c4d2ae640 1 Processor -- start 2024-04-25T13:30:20.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.308+0000 7f0c4d2ae640 1 -- start start 2024-04-25T13:30:20.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c4d2ae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c481031d0 0x7f0c481986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:20.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c4d2ae640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c48103af0 0x7f0c48198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:20.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c4d2ae640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c4810c510 0x7f0c4819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:20.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c4d2ae640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0c48112180 con 0x7f0c481031d0 2024-04-25T13:30:20.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c4d2ae640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c48112000 con 0x7f0c48103af0 2024-04-25T13:30:20.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c4d2ae640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0c48112300 con 0x7f0c4810c510 2024-04-25T13:30:20.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c46ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c481031d0 0x7f0c481986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:20.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c46ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c481031d0 0x7f0c481986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36382/0 (socket says 172.21.15.31:36382) 2024-04-25T13:30:20.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c46ffd640 1 -- 172.21.15.31:0/2199528099 learned_addr learned my addr 172.21.15.31:0/2199528099 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:20.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c3ffff640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c48103af0 0x7f0c48198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:20.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c477fe640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c4810c510 0x7f0c4819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:20.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c46ffd640 1 -- 172.21.15.31:0/2199528099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c4810c510 msgr2=0x7f0c4819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c46ffd640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c4810c510 0x7f0c4819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c46ffd640 1 -- 172.21.15.31:0/2199528099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c48103af0 msgr2=0x7f0c48198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c46ffd640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c48103af0 0x7f0c48198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c46ffd640 1 -- 172.21.15.31:0/2199528099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4819d6d0 con 0x7f0c481031d0 2024-04-25T13:30:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c477fe640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c4810c510 0x7f0c4819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:30:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c46ffd640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c481031d0 0x7f0c481986e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800c920 tx=0x7f0c3800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c44ff9640 1 -- 172.21.15.31:0/2199528099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38007c70 con 0x7f0c481031d0 2024-04-25T13:30:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/2199528099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4819d9c0 con 0x7f0c481031d0 2024-04-25T13:30:20.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c44ff9640 1 -- 172.21.15.31:0/2199528099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c38007e10 con 0x7f0c481031d0 2024-04-25T13:30:20.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c44ff9640 1 -- 172.21.15.31:0/2199528099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38018820 con 0x7f0c481031d0 2024-04-25T13:30:20.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.309+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/2199528099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c481a5250 con 0x7f0c481031d0 2024-04-25T13:30:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.310+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/2199528099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c48068d70 con 0x7f0c481031d0 2024-04-25T13:30:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.312+0000 7f0c44ff9640 1 -- 172.21.15.31:0/2199528099 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0c3800f450 con 0x7f0c481031d0 2024-04-25T13:30:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.313+0000 7f0c44ff9640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0c340a3d20 0x7f0c340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.313+0000 7f0c44ff9640 1 -- 172.21.15.31:0/2199528099 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8671+0+0 (secure 0 0 0) 0x7f0c380d0c60 con 0x7f0c481031d0 2024-04-25T13:30:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.313+0000 7f0c3ffff640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0c340a3d20 0x7f0c340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.313+0000 7f0c3ffff640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0c340a3d20 0x7f0c340a61e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c30031440 tx=0x7f0c30005b50 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:20.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.317+0000 7f0c44ff9640 1 -- 172.21.15.31:0/2199528099 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c38014030 con 0x7f0c481031d0 2024-04-25T13:30:20.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:20.506+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/2199528099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0c481076c0 con 0x7f0c481031d0 2024-04-25T13:30:21.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.185+0000 7f0c44ff9640 1 -- 172.21.15.31:0/2199528099 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7f0c3809d300 con 0x7f0c481031d0 2024-04-25T13:30:21.187 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:30:21.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/2199528099 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0c340a3d20 msgr2=0x7f0c340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:21.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0c340a3d20 0x7f0c340a61e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c30031440 tx=0x7f0c30005b50 comp rx=0 tx=0).stop 2024-04-25T13:30:21.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/2199528099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c481031d0 msgr2=0x7f0c481986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:21.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c481031d0 0x7f0c481986e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800c920 tx=0x7f0c3800cde0 comp rx=0 tx=0).stop 2024-04-25T13:30:21.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/2199528099 shutdown_connections 2024-04-25T13:30:21.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f0c340a3d20 0x7f0c340a61e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:21.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c4810c510 0x7f0c4819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:21.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c48103af0 0x7f0c48198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:21.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 --2- 172.21.15.31:0/2199528099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c481031d0 0x7f0c481986e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:21.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/2199528099 >> 172.21.15.31:0/2199528099 conn(0x7f0c48077f50 msgr2=0x7f0c480fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:21.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/2199528099 shutdown_connections 2024-04-25T13:30:21.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.189+0000 7f0c4d2ae640 1 -- 172.21.15.31:0/2199528099 wait complete. 2024-04-25T13:30:21.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:20 smithi031 ceph-mon[29457]: pgmap v341: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:30:21.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:20 smithi031 ceph-mon[29457]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:30:21.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:20 smithi031 ceph-mon[29457]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:30:21.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:20 smithi031 ceph-mon[29457]: Cluster is now healthy 2024-04-25T13:30:21.205 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:20 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/3886233648' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:30:21.205 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:20 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.136:6838/79299671,v1:172.21.15.136:6839/79299671] up:boot 2024-04-25T13:30:21.205 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:20 smithi031 ceph-mon[29457]: mds.? [v2:172.21.15.66:6838/2393874263,v1:172.21.15.66:6839/2393874263] up:boot 2024-04-25T13:30:21.205 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:20 smithi031 ceph-mon[29457]: fsmap 12 up:standby 2024-04-25T13:30:21.205 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:20 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:30:21.205 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:20 smithi031 ceph-mon[29457]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:30:21.205 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:20 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2199528099' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:30:21.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:30:21.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:20 smithi136 ceph-mon[33038]: pgmap v341: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:30:21.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:20 smithi136 ceph-mon[33038]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:30:21.219 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:20 smithi136 ceph-mon[33038]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:30:21.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:20 smithi136 ceph-mon[33038]: Cluster is now healthy 2024-04-25T13:30:21.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:20 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/3886233648' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:30:21.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:20 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.136:6838/79299671,v1:172.21.15.136:6839/79299671] up:boot 2024-04-25T13:30:21.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:20 smithi136 ceph-mon[33038]: mds.? [v2:172.21.15.66:6838/2393874263,v1:172.21.15.66:6839/2393874263] up:boot 2024-04-25T13:30:21.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:20 smithi136 ceph-mon[33038]: fsmap 12 up:standby 2024-04-25T13:30:21.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:20 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:30:21.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:20 smithi136 ceph-mon[33038]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:30:21.220 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:20 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2199528099' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:30:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 -- 172.21.15.31:0/206441210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a24106160 msgr2=0x7f6a2410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 --2- 172.21.15.31:0/206441210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a24106160 0x7f6a2410dd30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009900 tx=0x7f6a1c02a410 comp rx=0 tx=0).stop 2024-04-25T13:30:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 -- 172.21.15.31:0/206441210 shutdown_connections 2024-04-25T13:30:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 --2- 172.21.15.31:0/206441210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a24106160 0x7f6a2410dd30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 --2- 172.21.15.31:0/206441210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a24101a40 0x7f6a24105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 --2- 172.21.15.31:0/206441210 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a24101090 0x7f6a24101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 -- 172.21.15.31:0/206441210 >> 172.21.15.31:0/206441210 conn(0x7f6a24077f50 msgr2=0x7f6a24078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 -- 172.21.15.31:0/206441210 shutdown_connections 2024-04-25T13:30:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 -- 172.21.15.31:0/206441210 wait complete. 2024-04-25T13:30:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 Processor -- start 2024-04-25T13:30:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 -- start start 2024-04-25T13:30:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a24101090 0x7f6a241986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a24106160 0x7f6a24198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2419cf70 0x7f6a2419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241110b0 con 0x7f6a2419cf70 2024-04-25T13:30:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24110f30 con 0x7f6a24101090 2024-04-25T13:30:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.324+0000 7f6a2aca7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24111230 con 0x7f6a24106160 2024-04-25T13:30:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a29244640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2419cf70 0x7f6a2419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a29244640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2419cf70 0x7f6a2419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36398/0 (socket says 172.21.15.31:36398) 2024-04-25T13:30:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a29244640 1 -- 172.21.15.31:0/897450608 learned_addr learned my addr 172.21.15.31:0/897450608 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-25T13:30:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a28a43640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a24101090 0x7f6a241986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a23fff640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a24106160 0x7f6a24198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a29244640 1 -- 172.21.15.31:0/897450608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a24106160 msgr2=0x7f6a24198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a29244640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a24106160 0x7f6a24198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a29244640 1 -- 172.21.15.31:0/897450608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a24101090 msgr2=0x7f6a241986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a29244640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a24101090 0x7f6a241986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a29244640 1 -- 172.21.15.31:0/897450608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2419da60 con 0x7f6a2419cf70 2024-04-25T13:30:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a23fff640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a24106160 0x7f6a24198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:30:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a28a43640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a24101090 0x7f6a241986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:30:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a29244640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2419cf70 0x7f6a2419d420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c004770 tx=0x7f6a1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:21.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a21ffb640 1 -- 172.21.15.31:0/897450608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c03f030 con 0x7f6a2419cf70 2024-04-25T13:30:21.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a21ffb640 1 -- 172.21.15.31:0/897450608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a1c003aa0 con 0x7f6a2419cf70 2024-04-25T13:30:21.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a2aca7640 1 -- 172.21.15.31:0/897450608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2419dcf0 con 0x7f6a2419cf70 2024-04-25T13:30:21.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.325+0000 7f6a21ffb640 1 -- 172.21.15.31:0/897450608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c035070 con 0x7f6a2419cf70 2024-04-25T13:30:21.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.326+0000 7f6a2aca7640 1 -- 172.21.15.31:0/897450608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a241a5510 con 0x7f6a2419cf70 2024-04-25T13:30:21.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.326+0000 7f6a2aca7640 1 -- 172.21.15.31:0/897450608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a24068d70 con 0x7f6a2419cf70 2024-04-25T13:30:21.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.328+0000 7f6a21ffb640 1 -- 172.21.15.31:0/897450608 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6a1c042030 con 0x7f6a2419cf70 2024-04-25T13:30:21.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.328+0000 7f6a21ffb640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f6a100a3da0 0x7f6a100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:30:21.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.328+0000 7f6a28a43640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f6a100a3da0 0x7f6a100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:30:21.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.328+0000 7f6a21ffb640 1 -- 172.21.15.31:0/897450608 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8175+0+0 (secure 0 0 0) 0x7f6a1c0f0c00 con 0x7f6a2419cf70 2024-04-25T13:30:21.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.329+0000 7f6a28a43640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f6a100a3da0 0x7f6a100a6260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009fc0 tx=0x7f6a0c005f90 comp rx=0 tx=0).ready entity=mgr.24922 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:30:21.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.333+0000 7f6a21ffb640 1 -- 172.21.15.31:0/897450608 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a1c0f7080 con 0x7f6a2419cf70 2024-04-25T13:30:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:20 smithi066 ceph-mon[33137]: pgmap v341: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:30:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:20 smithi066 ceph-mon[33137]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:30:21.408 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:20 smithi066 ceph-mon[33137]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:30:21.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:20 smithi066 ceph-mon[33137]: Cluster is now healthy 2024-04-25T13:30:21.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:20 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/3886233648' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:30:21.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:20 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.136:6838/79299671,v1:172.21.15.136:6839/79299671] up:boot 2024-04-25T13:30:21.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:20 smithi066 ceph-mon[33137]: mds.? [v2:172.21.15.66:6838/2393874263,v1:172.21.15.66:6839/2393874263] up:boot 2024-04-25T13:30:21.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:20 smithi066 ceph-mon[33137]: fsmap 12 up:standby 2024-04-25T13:30:21.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:20 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:30:21.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:20 smithi066 ceph-mon[33137]: from='mgr.24922 172.21.15.31:0/3485665426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:30:21.409 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:20 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2199528099' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:30:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:21.523+0000 7f6a2aca7640 1 -- 172.21.15.31:0/897450608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6a24101a40 con 0x7f6a2419cf70 2024-04-25T13:30:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.191+0000 7f6a21ffb640 1 -- 172.21.15.31:0/897450608 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7f6a1c0bd450 con 0x7f6a2419cf70 2024-04-25T13:30:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-04-25T13:30:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 -- 172.21.15.31:0/897450608 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f6a100a3da0 msgr2=0x7f6a100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f6a100a3da0 0x7f6a100a6260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009fc0 tx=0x7f6a0c005f90 comp rx=0 tx=0).stop 2024-04-25T13:30:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 -- 172.21.15.31:0/897450608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2419cf70 msgr2=0x7f6a2419d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:30:22.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2419cf70 0x7f6a2419d420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c004770 tx=0x7f6a1c004270 comp rx=0 tx=0).stop 2024-04-25T13:30:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 -- 172.21.15.31:0/897450608 shutdown_connections 2024-04-25T13:30:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.31:6800/332946768,v1:172.21.15.31:6801/332946768] conn(0x7f6a100a3da0 0x7f6a100a6260 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a2419cf70 0x7f6a2419d420 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a24106160 0x7f6a24198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 --2- 172.21.15.31:0/897450608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a24101090 0x7f6a241986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:30:22.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 -- 172.21.15.31:0/897450608 >> 172.21.15.31:0/897450608 conn(0x7f6a24077f50 msgr2=0x7f6a24069410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:30:22.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 -- 172.21.15.31:0/897450608 shutdown_connections 2024-04-25T13:30:22.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-25T13:30:22.194+0000 7f6a2aca7640 1 -- 172.21.15.31:0/897450608 wait complete. 2024-04-25T13:30:22.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:30:22.240 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:30:22.273 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:30:22.305 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T13:30:22.305 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:30:22.322 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:30:22.340 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:30:22.361 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T13:30:22.362 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T13:30:22.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.a 2024-04-25T13:30:22.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:22 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/2199528099' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:30:22.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:22 smithi136 ceph-mon[33038]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T13:30:22.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:22 smithi136 ceph-mon[33038]: pgmap v343: 65 pgs: 65 active+clean; 577 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:30:22.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:22 smithi136 ceph-mon[33038]: from='client.? 172.21.15.31:0/897450608' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:30:22.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:22 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/2199528099' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:30:22.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:22 smithi031 ceph-mon[29457]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T13:30:22.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:22 smithi031 ceph-mon[29457]: pgmap v343: 65 pgs: 65 active+clean; 577 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:30:22.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:22 smithi031 ceph-mon[29457]: from='client.? 172.21.15.31:0/897450608' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:30:22.484 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:22 smithi031 systemd[1]: Stopping Ceph mon.a for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:30:22.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:22 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/2199528099' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:30:22.658 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:22 smithi066 ceph-mon[33137]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T13:30:22.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:22 smithi066 ceph-mon[33137]: pgmap v343: 65 pgs: 65 active+clean; 577 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:30:22.659 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:22 smithi066 ceph-mon[33137]: from='client.? 172.21.15.31:0/897450608' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:30:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:22 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a[29453]: 2024-04-25T13:30:22.659+0000 7f59fde8c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:30:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:22 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a[29453]: 2024-04-25T13:30:22.659+0000 7f59fde8c700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T13:30:22.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:22 smithi031 podman[86521]: 2024-04-25 13:30:22.743153154 +0000 UTC m=+0.178267308 container died ab96c980d7ccb9e039321d7aeee40da8501cbb9cac416864020873e24024f74f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:30:23.361 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:23 smithi031 podman[86521]: 2024-04-25 13:30:23.101948225 +0000 UTC m=+0.537062388 container cleanup ab96c980d7ccb9e039321d7aeee40da8501cbb9cac416864020873e24024f74f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:30:23.361 INFO:journalctl@ceph.mon.a.smithi031.stdout:Apr 25 13:30:23 smithi031 bash[86521]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-a 2024-04-25T13:30:23.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T13:30:23.577 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:30:23.578 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T13:30:23.578 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T13:30:23.578 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.b 2024-04-25T13:30:23.889 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:23 smithi066 systemd[1]: Stopping Ceph mon.b for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:30:24.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:23 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-b[33133]: 2024-04-25T13:30:23.887+0000 7f50dd26a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:30:24.158 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:23 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-b[33133]: 2024-04-25T13:30:23.887+0000 7f50dd26a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T13:30:24.159 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:23 smithi066 podman[68964]: 2024-04-25 13:30:23.949390635 +0000 UTC m=+0.172224874 container died f9fc2a020c46b446c917ccbc7872a9f4faa45eeb173d85232b55916b113f079d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:30:24.588 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:24 smithi066 podman[68964]: 2024-04-25 13:30:24.30417332 +0000 UTC m=+0.527007565 container cleanup f9fc2a020c46b446c917ccbc7872a9f4faa45eeb173d85232b55916b113f079d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T13:30:24.589 INFO:journalctl@ceph.mon.b.smithi066.stdout:Apr 25 13:30:24 smithi066 bash[68964]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-b 2024-04-25T13:30:24.722 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T13:30:24.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:30:24.765 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T13:30:24.765 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T13:30:24.765 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.c 2024-04-25T13:30:25.087 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:24 smithi136 systemd[1]: Stopping Ceph mon.c for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:30:25.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:25 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-c[33034]: 2024-04-25T13:30:25.086+0000 7f5202535700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:30:25.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:25 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-c[33034]: 2024-04-25T13:30:25.086+0000 7f5202535700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T13:30:25.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:25 smithi136 podman[68917]: 2024-04-25 13:30:25.250041363 +0000 UTC m=+0.289929556 container died 730fc5357bb7cea3d674c0119bc5827f5ea299165bf489f05fe844b04ea5bae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T13:30:25.880 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:25 smithi136 podman[68917]: 2024-04-25 13:30:25.653890113 +0000 UTC m=+0.693778305 container cleanup 730fc5357bb7cea3d674c0119bc5827f5ea299165bf489f05fe844b04ea5bae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T13:30:25.880 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:25 smithi136 bash[68917]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-c 2024-04-25T13:30:26.131 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 13:30:25 smithi136 podman[68931]: 2024-04-25 13:30:25.879848831 +0000 UTC m=+0.630776931 container remove 730fc5357bb7cea3d674c0119bc5827f5ea299165bf489f05fe844b04ea5bae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mon-c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T13:30:26.136 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T13:30:26.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:30:26.183 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T13:30:26.183 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T13:30:26.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.x 2024-04-25T13:30:26.484 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:30:26 smithi031 systemd[1]: Stopping Ceph mgr.x for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:30:26.781 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:30:26 smithi031 podman[86643]: 2024-04-25 13:30:26.520496968 +0000 UTC m=+0.137938026 container died 8f9a4fffd858606d910d0c8c6d46a60371af602a5893b7a5847f1ee2214f4ed1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:30:27.091 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:30:26 smithi031 podman[86643]: 2024-04-25 13:30:26.864294739 +0000 UTC m=+0.481735790 container cleanup 8f9a4fffd858606d910d0c8c6d46a60371af602a5893b7a5847f1ee2214f4ed1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T13:30:27.091 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:30:26 smithi031 bash[86643]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x 2024-04-25T13:30:27.341 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:30:27 smithi031 podman[86657]: 2024-04-25 13:30:27.090743671 +0000 UTC m=+0.566586460 container remove 8f9a4fffd858606d910d0c8c6d46a60371af602a5893b7a5847f1ee2214f4ed1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:30:27.341 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:30:27 smithi031 systemd[1]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.x.service: Deactivated successfully. 2024-04-25T13:30:27.341 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Apr 25 13:30:27 smithi031 systemd[1]: Stopped Ceph mgr.x for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:30:27.343 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T13:30:27.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:30:27.378 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T13:30:27.378 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T13:30:27.378 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.y 2024-04-25T13:30:27.658 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:30:27 smithi066 systemd[1]: Stopping Ceph mgr.y for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:30:27.984 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:30:27 smithi066 podman[69086]: 2024-04-25 13:30:27.713120658 +0000 UTC m=+0.141003317 container died 13a7b580e9ef0f03f5368820fe702b145d9e36db3a9e78901ca9faca950fc39b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:30:28.276 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:30:28 smithi066 podman[69086]: 2024-04-25 13:30:28.058827707 +0000 UTC m=+0.486710360 container cleanup 13a7b580e9ef0f03f5368820fe702b145d9e36db3a9e78901ca9faca950fc39b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T13:30:28.277 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:30:28 smithi066 bash[69086]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y 2024-04-25T13:30:28.531 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:30:28 smithi066 podman[69100]: 2024-04-25 13:30:28.276702478 +0000 UTC m=+0.559196944 container remove 13a7b580e9ef0f03f5368820fe702b145d9e36db3a9e78901ca9faca950fc39b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:30:28.532 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Apr 25 13:30:28 smithi066 systemd[1]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-25T13:30:28.535 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T13:30:28.571 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:30:28.571 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T13:30:28.571 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T13:30:28.572 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.z 2024-04-25T13:30:28.911 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:30:28 smithi136 systemd[1]: Stopping Ceph mgr.z for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:30:29.199 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:30:28 smithi136 podman[69040]: 2024-04-25 13:30:28.911627115 +0000 UTC m=+0.141172760 container died 2d8062f9d8e8f6a3bc004d9e4974bee78fd677e383dcf3a0ed5a817b8b86f752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T13:30:29.469 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:30:29 smithi136 podman[69040]: 2024-04-25 13:30:29.282155096 +0000 UTC m=+0.511700728 container cleanup 2d8062f9d8e8f6a3bc004d9e4974bee78fd677e383dcf3a0ed5a817b8b86f752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:30:29.469 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 13:30:29 smithi136 bash[69040]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mgr-z 2024-04-25T13:30:29.676 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T13:30:29.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:30:29.710 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T13:30:29.711 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T13:30:29.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.0 2024-04-25T13:30:30.036 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:29 smithi031 systemd[1]: Stopping Ceph osd.0 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:30:30.484 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0[44419]: 2024-04-25T13:30:30.134+0000 7f8edb4d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:30:30.484 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0[44419]: 2024-04-25T13:30:30.134+0000 7f8edb4d9700 -1 osd.0 109 *** Got signal Terminated *** 2024-04-25T13:30:30.484 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0[44419]: 2024-04-25T13:30:30.134+0000 7f8edb4d9700 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:30:35.969 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:35 smithi031 podman[86765]: 2024-04-25 13:30:35.659757381 +0000 UTC m=+5.750994078 container died 2d533a8dd4aee389698d48716d22c5344f12bfaebbafd8deb06474444242d479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T13:30:36.234 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:36 smithi031 podman[86765]: 2024-04-25 13:30:36.026201986 +0000 UTC m=+6.117438683 container cleanup 2d533a8dd4aee389698d48716d22c5344f12bfaebbafd8deb06474444242d479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:30:36.234 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:36 smithi031 bash[86765]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0 2024-04-25T13:30:36.602 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:36 smithi031 podman[86791]: 2024-04-25 13:30:36.268944622 +0000 UTC m=+0.602678879 container remove 2d533a8dd4aee389698d48716d22c5344f12bfaebbafd8deb06474444242d479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T13:30:36.930 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:36 smithi031 podman[86882]: 2024-04-25 13:30:36.602941564 +0000 UTC m=+0.104955708 container create 8a6555a194617a509d25df35f39690c792d387d3a2e00fdbbdfa059493585a68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T13:30:36.931 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:36 smithi031 podman[86882]: 2024-04-25 13:30:36.516648966 +0000 UTC m=+0.018663112 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:30:36.931 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:36 smithi031 podman[86882]: 2024-04-25 13:30:36.830185418 +0000 UTC m=+0.332199562 container init 8a6555a194617a509d25df35f39690c792d387d3a2e00fdbbdfa059493585a68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T13:30:36.931 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:36 smithi031 podman[86882]: 2024-04-25 13:30:36.833468483 +0000 UTC m=+0.335482627 container start 8a6555a194617a509d25df35f39690c792d387d3a2e00fdbbdfa059493585a68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:30:36.931 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:36 smithi031 podman[86882]: 2024-04-25 13:30:36.930296902 +0000 UTC m=+0.432311050 container attach 8a6555a194617a509d25df35f39690c792d387d3a2e00fdbbdfa059493585a68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T13:30:37.234 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 25 13:30:37 smithi031 podman[86882]: 2024-04-25 13:30:37.135743099 +0000 UTC m=+0.637757235 container died 8a6555a194617a509d25df35f39690c792d387d3a2e00fdbbdfa059493585a68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T13:30:37.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T13:30:37.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:30:37.755 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T13:30:37.755 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T13:30:37.755 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.1 2024-04-25T13:30:38.071 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:37 smithi066 systemd[1]: Stopping Ceph osd.1 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:30:38.408 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:38 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1[37750]: 2024-04-25T13:30:38.070+0000 7efd8a1df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:30:38.408 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:38 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1[37750]: 2024-04-25T13:30:38.070+0000 7efd8a1df700 -1 osd.1 109 *** Got signal Terminated *** 2024-04-25T13:30:38.408 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:38 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1[37750]: 2024-04-25T13:30:38.070+0000 7efd8a1df700 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:30:43.815 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:43 smithi066 podman[69215]: 2024-04-25 13:30:43.541577104 +0000 UTC m=+5.580189116 container died f88a5db2c0c6283d6e7e6d90e924920c34103d58a63f8c0ec56548a560f45434 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T13:30:44.158 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:43 smithi066 podman[69215]: 2024-04-25 13:30:43.913427372 +0000 UTC m=+5.952039374 container cleanup f88a5db2c0c6283d6e7e6d90e924920c34103d58a63f8c0ec56548a560f45434 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T13:30:44.159 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:43 smithi066 bash[69215]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1 2024-04-25T13:30:44.543 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:44 smithi066 podman[69232]: 2024-04-25 13:30:44.189997717 +0000 UTC m=+0.643993696 container remove f88a5db2c0c6283d6e7e6d90e924920c34103d58a63f8c0ec56548a560f45434 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T13:30:44.827 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:44 smithi066 podman[69326]: 2024-04-25 13:30:44.446532205 +0000 UTC m=+0.018698150 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:30:44.827 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:44 smithi066 podman[69326]: 2024-04-25 13:30:44.549665146 +0000 UTC m=+0.121831083 container create 0bc8d21d896fc2d9780ea6ecb0ce3aa33aef7cf9c47effe85153ccfee5155454 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T13:30:44.827 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:44 smithi066 podman[69326]: 2024-04-25 13:30:44.743567489 +0000 UTC m=+0.315733427 container init 0bc8d21d896fc2d9780ea6ecb0ce3aa33aef7cf9c47effe85153ccfee5155454 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:30:44.827 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:44 smithi066 podman[69326]: 2024-04-25 13:30:44.746682962 +0000 UTC m=+0.318848902 container start 0bc8d21d896fc2d9780ea6ecb0ce3aa33aef7cf9c47effe85153ccfee5155454 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T13:30:44.827 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:44 smithi066 podman[69326]: 2024-04-25 13:30:44.826725197 +0000 UTC m=+0.398891145 container attach 0bc8d21d896fc2d9780ea6ecb0ce3aa33aef7cf9c47effe85153ccfee5155454 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T13:30:45.158 INFO:journalctl@ceph.osd.1.smithi066.stdout:Apr 25 13:30:45 smithi066 podman[69326]: 2024-04-25 13:30:45.047871203 +0000 UTC m=+0.620037144 container died 0bc8d21d896fc2d9780ea6ecb0ce3aa33aef7cf9c47effe85153ccfee5155454 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:30:45.627 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T13:30:45.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:30:45.667 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T13:30:45.667 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T13:30:45.668 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.2 2024-04-25T13:30:45.969 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:45 smithi136 systemd[1]: Stopping Ceph osd.2 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:30:46.469 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2[38228]: 2024-04-25T13:30:45.975+0000 7f1e44cd6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:30:46.469 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2[38228]: 2024-04-25T13:30:45.975+0000 7f1e44cd6700 -1 osd.2 109 *** Got signal Terminated *** 2024-04-25T13:30:46.469 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2[38228]: 2024-04-25T13:30:45.975+0000 7f1e44cd6700 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:30:51.677 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:51 smithi136 podman[69177]: 2024-04-25 13:30:51.40634657 +0000 UTC m=+5.537168222 container died 41d5fe0510ac5c551f53dfb363948dca0d7029188ca7d2ad60c4f110549720dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T13:30:51.935 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:51 smithi136 podman[69177]: 2024-04-25 13:30:51.759479092 +0000 UTC m=+5.890300738 container cleanup 41d5fe0510ac5c551f53dfb363948dca0d7029188ca7d2ad60c4f110549720dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:30:51.936 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:51 smithi136 bash[69177]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2 2024-04-25T13:30:52.219 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:51 smithi136 podman[69194]: 2024-04-25 13:30:51.935608351 +0000 UTC m=+0.523404271 container remove 41d5fe0510ac5c551f53dfb363948dca0d7029188ca7d2ad60c4f110549720dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T13:30:52.705 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:52 smithi136 podman[69283]: 2024-04-25 13:30:52.377862124 +0000 UTC m=+0.111247429 container create 9bfeb6b1a1d267febe3f46363e0783c37aa0ea60974e782c5d3145e81f5ed743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:30:52.705 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:52 smithi136 podman[69283]: 2024-04-25 13:30:52.284598608 +0000 UTC m=+0.017983934 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:30:52.706 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:52 smithi136 podman[69283]: 2024-04-25 13:30:52.621781379 +0000 UTC m=+0.355166680 container init 9bfeb6b1a1d267febe3f46363e0783c37aa0ea60974e782c5d3145e81f5ed743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T13:30:52.706 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:52 smithi136 podman[69283]: 2024-04-25 13:30:52.624756095 +0000 UTC m=+0.358141400 container start 9bfeb6b1a1d267febe3f46363e0783c37aa0ea60974e782c5d3145e81f5ed743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:30:52.706 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:52 smithi136 podman[69283]: 2024-04-25 13:30:52.705002721 +0000 UTC m=+0.438388036 container attach 9bfeb6b1a1d267febe3f46363e0783c37aa0ea60974e782c5d3145e81f5ed743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T13:30:52.969 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 13:30:52 smithi136 podman[69283]: 2024-04-25 13:30:52.926389069 +0000 UTC m=+0.659774373 container died 9bfeb6b1a1d267febe3f46363e0783c37aa0ea60974e782c5d3145e81f5ed743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T13:30:53.513 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T13:30:53.553 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:30:53.554 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T13:30:53.554 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T13:30:53.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.3 2024-04-25T13:30:53.852 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:30:53 smithi031 systemd[1]: Stopping Ceph osd.3 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:30:54.234 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:30:53 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3[50953]: 2024-04-25T13:30:53.852+0000 7f6b7ad97700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:30:54.234 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:30:53 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3[50953]: 2024-04-25T13:30:53.852+0000 7f6b7ad97700 -1 osd.3 109 *** Got signal Terminated *** 2024-04-25T13:30:54.235 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:30:53 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3[50953]: 2024-04-25T13:30:53.852+0000 7f6b7ad97700 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:30:56.819 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:30:56 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:30:56.534+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:30:56.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:30:56 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:30:56.692+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:30:57.440 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:30:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:30:56.969+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:30:57.719 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:30:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:30:57.440+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:30:57.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:30:57 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:30:57.486+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:30:57.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:30:57 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:30:57.720+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:30:58.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:30:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:30:57.959+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:30:58.719 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:30:58 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:30:58.467+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:30:58.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:30:58 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:30:58.495+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:30:58.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:30:58 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:30:58.717+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:30:59.223 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:30:58 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:30:58.921+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:30:59.484 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:30:59 smithi031 podman[86990]: 2024-04-25 13:30:59.192207837 +0000 UTC m=+5.426259302 container died 1ea286ad3e4be663e6ee0771d34f500353b29750db9ba02cdb79339552ff7e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:30:59.764 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:30:59 smithi031 podman[86990]: 2024-04-25 13:30:59.567944337 +0000 UTC m=+5.801995801 container cleanup 1ea286ad3e4be663e6ee0771d34f500353b29750db9ba02cdb79339552ff7e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:30:59.764 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:30:59 smithi031 bash[86990]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3 2024-04-25T13:30:59.903 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:30:59 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:30:59.498+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:30:59.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:30:59 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:30:59.518+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:00.199 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:30:59 smithi031 podman[87009]: 2024-04-25 13:30:59.8274038 +0000 UTC m=+0.626578219 container remove 1ea286ad3e4be663e6ee0771d34f500353b29750db9ba02cdb79339552ff7e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T13:31:00.199 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:30:59 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:30:59.763+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:00.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:30:59 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:30:59.903+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:00.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:00 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:00.292+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:00.485 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:31:00 smithi031 podman[87100]: 2024-04-25 13:31:00.102382348 +0000 UTC m=+0.018625884 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:31:00.485 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:31:00 smithi031 podman[87100]: 2024-04-25 13:31:00.228316583 +0000 UTC m=+0.144560091 container create 052fa7167ed9d90c3e6445536706a5438d33cc4813180582bb412ebe92a5ba8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T13:31:00.485 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:31:00 smithi031 podman[87100]: 2024-04-25 13:31:00.438914746 +0000 UTC m=+0.355158286 container init 052fa7167ed9d90c3e6445536706a5438d33cc4813180582bb412ebe92a5ba8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T13:31:00.485 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:31:00 smithi031 podman[87100]: 2024-04-25 13:31:00.44202712 +0000 UTC m=+0.358270642 container start 052fa7167ed9d90c3e6445536706a5438d33cc4813180582bb412ebe92a5ba8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:31:00.750 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:31:00 smithi031 podman[87100]: 2024-04-25 13:31:00.547014041 +0000 UTC m=+0.463257557 container attach 052fa7167ed9d90c3e6445536706a5438d33cc4813180582bb412ebe92a5ba8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:31:00.876 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:00 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:00.500+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:00.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:00 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:00.487+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:01.050 INFO:journalctl@ceph.osd.3.smithi031.stdout:Apr 25 13:31:00 smithi031 podman[87100]: 2024-04-25 13:31:00.750887559 +0000 UTC m=+0.667131086 container died 052fa7167ed9d90c3e6445536706a5438d33cc4813180582bb412ebe92a5ba8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T13:31:01.050 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:00 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:00.783+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:01.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:00 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:00.876+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:01.358 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:01 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:01.295+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:01.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T13:31:01.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:31:01.427 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T13:31:01.427 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T13:31:01.428 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.4 2024-04-25T13:31:01.742 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:01 smithi066 systemd[1]: Stopping Ceph osd.4 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:31:01.743 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:01 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:01.491+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:01.835 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:01 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:01.508+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:01.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:01 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:01.741+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:02.158 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:01 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4[44554]: 2024-04-25T13:31:01.742+0000 7f72e821e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:31:02.158 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:01 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4[44554]: 2024-04-25T13:31:01.742+0000 7f72e821e700 -1 osd.4 109 *** Got signal Terminated *** 2024-04-25T13:31:02.159 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:01 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4[44554]: 2024-04-25T13:31:01.742+0000 7f72e821e700 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:31:02.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:01 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:01.833+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:02.614 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:02 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:02.336+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:02.814 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:02 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:02.548+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:02.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:02 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:02.508+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:02.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:02 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:02.711+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:03.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:02 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:02.813+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:03.664 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:03 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:03.346+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:03.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:03 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:03.523+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:03.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:03 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:03.556+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:03.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:03 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:03.664+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:04.193 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:03 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:03.847+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:04.622 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:04 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:04.305+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:04.847 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:04 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:04.556+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:04.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:04 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:04.529+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:04.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:04 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:04.621+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:05.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:04 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:04.846+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:05.599 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:05 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:05.257+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:05.600 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:05 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:05.257+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:05.859 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:05 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:05.572+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:05.890 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:05 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4[44554]: 2024-04-25T13:31:05.622+0000 7f72e080a700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:37.326513+0000 front 2024-04-25T13:30:37.326473+0000 (oldest deadline 2024-04-25T13:31:05.527131+0000) 2024-04-25T13:31:05.890 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:05 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:05.562+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:05.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:05 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:05.599+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:06.158 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:05 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:05.889+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:06.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:05 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:05.858+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:06.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:06 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:06.209+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:06.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:06 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:06.209+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:06.837 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:06 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:06.580+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:06.908 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:06 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4[44554]: 2024-04-25T13:31:06.598+0000 7f72e080a700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:37.326513+0000 front 2024-04-25T13:30:37.326473+0000 (oldest deadline 2024-04-25T13:31:05.527131+0000) 2024-04-25T13:31:06.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:06 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:06.541+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:06.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:06 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:06.541+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:06.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:06 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:06.586+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:07.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:06 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:06.836+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:07.408 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:07 smithi066 podman[69456]: 2024-04-25 13:31:07.172127764 +0000 UTC m=+5.540848413 container died 9558c3c8657312a732685cb9a377014cd2b66e9efb203387fded1e6a8387a3b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T13:31:07.409 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:06 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:06.932+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:07.561 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:07 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:07.247+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:07.561 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:07 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:07.247+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:07.819 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:07 smithi066 podman[69456]: 2024-04-25 13:31:07.534470948 +0000 UTC m=+5.903191597 container cleanup 9558c3c8657312a732685cb9a377014cd2b66e9efb203387fded1e6a8387a3b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:31:07.819 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:07 smithi066 bash[69456]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4 2024-04-25T13:31:07.820 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:07 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:07.562+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:07.820 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:07 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:07.562+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:07.969 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:07 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:07.580+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:07.969 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:07 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:07.813+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:07.969 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:07 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:07.813+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:07.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:07 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:07.560+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:08.158 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:07 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:07.889+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:08.159 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:07 smithi066 podman[69474]: 2024-04-25 13:31:07.819467963 +0000 UTC m=+0.646215747 container remove 9558c3c8657312a732685cb9a377014cd2b66e9efb203387fded1e6a8387a3b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T13:31:08.418 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:08 smithi066 podman[69568]: 2024-04-25 13:31:08.065000845 +0000 UTC m=+0.018241235 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:31:08.418 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:08 smithi066 podman[69568]: 2024-04-25 13:31:08.298095473 +0000 UTC m=+0.251335905 container create 9d933796123a3a72d215966c753ef4db2614c4b6b9da5f2ad62b7380cbdf5fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T13:31:08.558 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:08 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:08.240+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:08.578 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:08.246+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:08.578 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:08.246+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:08.814 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:08 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:08.571+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:08.815 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:08 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:08.571+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:08.815 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:08 smithi066 podman[69568]: 2024-04-25 13:31:08.508718657 +0000 UTC m=+0.461959033 container init 9d933796123a3a72d215966c753ef4db2614c4b6b9da5f2ad62b7380cbdf5fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:31:08.815 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:08 smithi066 podman[69568]: 2024-04-25 13:31:08.511939775 +0000 UTC m=+0.465180160 container start 9d933796123a3a72d215966c753ef4db2614c4b6b9da5f2ad62b7380cbdf5fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:31:08.815 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:08 smithi066 podman[69568]: 2024-04-25 13:31:08.591919242 +0000 UTC m=+0.545159624 container attach 9d933796123a3a72d215966c753ef4db2614c4b6b9da5f2ad62b7380cbdf5fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-04-25T13:31:08.822 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:08 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:08.557+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:08.822 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:08 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:08.557+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.945133+0000 front 2024-04-25T13:30:41.945326+0000 (oldest deadline 2024-04-25T13:31:07.845147+0000) 2024-04-25T13:31:08.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:08.578+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:08.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:08 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:08.578+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:09.094 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:08 smithi066 podman[69568]: 2024-04-25 13:31:08.814768578 +0000 UTC m=+0.768008958 container died 9d933796123a3a72d215966c753ef4db2614c4b6b9da5f2ad62b7380cbdf5fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-04-25T13:31:09.095 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:08 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:08.913+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:09.151 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:08 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:08.821+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:09.151 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:08 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:08.821+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:09.350 INFO:journalctl@ceph.osd.4.smithi066.stdout:Apr 25 13:31:09 smithi066 podman[69594]: 2024-04-25 13:31:09.336852081 +0000 UTC m=+0.515730415 container remove 9d933796123a3a72d215966c753ef4db2614c4b6b9da5f2ad62b7380cbdf5fd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:31:09.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T13:31:09.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:31:09.397 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T13:31:09.397 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T13:31:09.397 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.5 2024-04-25T13:31:09.419 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:09 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:09.194+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:09.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:09 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:09.197+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:09.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:09 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:09.197+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:09.658 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:09 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:09.556+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:09.658 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:09 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:09.556+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:09.701 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:09 smithi136 systemd[1]: Stopping Ceph osd.5 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:31:09.702 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:09 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:09.550+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:09.702 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:09 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:09.550+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.945133+0000 front 2024-04-25T13:30:41.945326+0000 (oldest deadline 2024-04-25T13:31:07.845147+0000) 2024-04-25T13:31:09.969 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:09 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:09.842+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:09.969 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:09 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:09.842+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:09.969 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:09 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:09.701+0000 7f81701e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:31:09.970 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:09 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:09.701+0000 7f81701e2700 -1 osd.5 109 *** Got signal Terminated *** 2024-04-25T13:31:09.970 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:09 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:09.701+0000 7f81701e2700 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:31:09.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:09 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:09.617+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:09.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:09 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:09.617+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:10.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:09 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:09.954+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:10.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:10 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:10.156+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:10.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:10 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:10.219+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:10.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:10 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:10.219+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:10.854 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:10 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:10.539+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:10.855 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:10 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:10.539+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.945133+0000 front 2024-04-25T13:30:41.945326+0000 (oldest deadline 2024-04-25T13:31:07.845147+0000) 2024-04-25T13:31:10.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:10 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:10.587+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:10.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:10 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:10.587+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:10.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:10 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:10.614+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:10.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:10 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:10.614+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:11.112 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:10 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:10.854+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:11.112 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:10 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:10.854+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:11.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:10 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:10.996+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:11.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:11 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:11.111+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:11.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:11 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:11.207+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:11.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:11 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:11.207+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:11.897 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:11 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:11.574+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:11.897 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:11 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:11.574+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.945133+0000 front 2024-04-25T13:30:41.945326+0000 (oldest deadline 2024-04-25T13:31:07.845147+0000) 2024-04-25T13:31:11.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:11 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:11.558+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:11.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:11 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:11.558+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:11.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:11 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:11.594+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:11.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:11 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:11.594+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:12.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:11 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:11.896+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:12.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:11 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:11.896+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:12.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:12 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:12.116+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:12.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:12 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:12.116+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:12.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:12 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:12.033+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:12.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:12 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:12.169+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:12.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:12 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:12.169+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:12.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:12 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:12.169+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.063774+0000 front 2024-04-25T13:30:47.063867+0000 (oldest deadline 2024-04-25T13:31:11.763703+0000) 2024-04-25T13:31:12.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:12 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:12.595+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:12.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:12 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:12.595+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:12.944 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:12 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:12.601+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:12.944 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:12 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:12.601+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.945133+0000 front 2024-04-25T13:30:41.945326+0000 (oldest deadline 2024-04-25T13:31:07.845147+0000) 2024-04-25T13:31:12.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:12 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:12.580+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:12.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:12 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:12.580+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:13.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:12 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:12.943+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:13.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:12 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:12.943+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:13.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:13 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:13.083+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:13.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:13 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:13.083+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:13.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:12 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:12.986+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:13.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:13 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:13.194+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:13.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:13 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:13.194+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:13.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:13 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:13.194+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.063774+0000 front 2024-04-25T13:30:47.063867+0000 (oldest deadline 2024-04-25T13:31:11.763703+0000) 2024-04-25T13:31:13.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:13 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:13.620+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:13.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:13 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:13.620+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:13.928 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:13 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:13.632+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:13.928 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:13 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:13.632+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.945133+0000 front 2024-04-25T13:30:41.945326+0000 (oldest deadline 2024-04-25T13:31:07.845147+0000) 2024-04-25T13:31:13.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:13 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:13.609+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:13.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:13 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:13.609+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:14.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:13 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:13.927+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:14.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:13 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:13.927+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:14.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:14 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:14.094+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:14.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:14 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:14.094+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:14.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:13 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:13.974+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:14.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:13 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:13.974+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:14.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:14 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:14.204+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:14.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:14 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:14.204+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:14.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:14 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:14.204+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.063774+0000 front 2024-04-25T13:30:47.063867+0000 (oldest deadline 2024-04-25T13:31:11.763703+0000) 2024-04-25T13:31:14.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:14 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:14.602+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:14.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:14 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:14.602+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:14.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:14 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:14.602+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:14.912 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:14 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:14.651+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:32.143013+0000 front 2024-04-25T13:30:32.142998+0000 (oldest deadline 2024-04-25T13:30:56.843068+0000) 2024-04-25T13:31:14.912 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:14 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5[44893]: 2024-04-25T13:31:14.651+0000 7f81687ce700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.945133+0000 front 2024-04-25T13:30:41.945326+0000 (oldest deadline 2024-04-25T13:31:07.845147+0000) 2024-04-25T13:31:14.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:14 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:14.653+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:14.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:14 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:14.653+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:14.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:14 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:14.653+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:15.219 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:15 smithi136 podman[69411]: 2024-04-25 13:31:15.043529871 +0000 UTC m=+5.466172018 container died 3a99c1bee720a706d67f71b84d1e86cd360f66c438c11ea17f62f2b5c9d3c5ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T13:31:15.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:14 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:14.911+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:15.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:14 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:14.911+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:15.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:15 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:15.114+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:15.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:15 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:15.114+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:15.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:14 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:14.936+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:15.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:14 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:14.936+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:15.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:15 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:15.216+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:15.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:15 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:15.216+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:15.484 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:15 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:15.216+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.063774+0000 front 2024-04-25T13:30:47.063867+0000 (oldest deadline 2024-04-25T13:31:11.763703+0000) 2024-04-25T13:31:15.719 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:15 smithi136 podman[69411]: 2024-04-25 13:31:15.542265393 +0000 UTC m=+5.964907543 container cleanup 3a99c1bee720a706d67f71b84d1e86cd360f66c438c11ea17f62f2b5c9d3c5ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:31:15.719 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:15 smithi136 bash[69411]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5 2024-04-25T13:31:15.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:15 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:15.629+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:15.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:15 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:15.629+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:15.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:15 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:15.629+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:15.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:15 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:15.667+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:15.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:15 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:15.667+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:15.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:15 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:15.667+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:16.102 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:15 smithi136 podman[69428]: 2024-04-25 13:31:15.776677754 +0000 UTC m=+0.726691553 container remove 3a99c1bee720a706d67f71b84d1e86cd360f66c438c11ea17f62f2b5c9d3c5ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:31:16.102 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:15 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:15.945+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:16.103 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:15 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:15.945+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:16.103 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:15 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:15.945+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:16.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:15 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:15.915+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:16.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:15 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:15.915+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:16.413 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:16 smithi136 podman[69519]: 2024-04-25 13:31:16.102369671 +0000 UTC m=+0.106501512 container create 89257f61b887112f4fff854a4e4ce626d75c6685cdd71b4633007424a77891b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T13:31:16.414 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:16 smithi136 podman[69519]: 2024-04-25 13:31:16.013495299 +0000 UTC m=+0.017627152 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:31:16.414 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:16 smithi136 podman[69519]: 2024-04-25 13:31:16.329840211 +0000 UTC m=+0.333972052 container init 89257f61b887112f4fff854a4e4ce626d75c6685cdd71b4633007424a77891b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:31:16.414 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:16 smithi136 podman[69519]: 2024-04-25 13:31:16.333045541 +0000 UTC m=+0.337177383 container start 89257f61b887112f4fff854a4e4ce626d75c6685cdd71b4633007424a77891b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T13:31:16.414 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:16 smithi136 podman[69519]: 2024-04-25 13:31:16.412901157 +0000 UTC m=+0.417033007 container attach 89257f61b887112f4fff854a4e4ce626d75c6685cdd71b4633007424a77891b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-04-25T13:31:16.415 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:16 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:16.155+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:16.415 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:16 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:16.155+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:16.658 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:16 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:16.251+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:16.658 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:16 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:16.251+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:16.658 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:16 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:16.251+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.063774+0000 front 2024-04-25T13:30:47.063867+0000 (oldest deadline 2024-04-25T13:31:11.763703+0000) 2024-04-25T13:31:16.720 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 13:31:16 smithi136 podman[69519]: 2024-04-25 13:31:16.628412111 +0000 UTC m=+0.632543955 container died 89257f61b887112f4fff854a4e4ce626d75c6685cdd71b4633007424a77891b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:31:16.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:16 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:16.595+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:16.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:16 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:16.595+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:16.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:16 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:16.595+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:16.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:16 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:16.657+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:16.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:16 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:16.657+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:16.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:16 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:16.657+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:17.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:17 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:17.159+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:17.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:17 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:17.159+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:17.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:16 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:16.912+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:17.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:16 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:16.913+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:17.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:16 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:16.913+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:17.290 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T13:31:17.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:31:17.369 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T13:31:17.369 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T13:31:17.370 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.6 2024-04-25T13:31:17.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:16 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:16.952+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:17.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:16 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:16.952+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:17.657 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:17 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:17.286+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:17.658 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:17 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:17.286+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:17.658 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:17 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:17.286+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.063774+0000 front 2024-04-25T13:30:47.063867+0000 (oldest deadline 2024-04-25T13:31:11.763703+0000) 2024-04-25T13:31:17.658 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:17 smithi031 systemd[1]: Stopping Ceph osd.6 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:31:17.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:17 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:17.572+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:17.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:17 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:17.572+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:17.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:17 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:17.572+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:17.984 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:17 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:17.676+0000 7fa9e2c7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:31:17.984 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:17 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:17.676+0000 7fa9e2c7e700 -1 osd.6 109 *** Got signal Terminated *** 2024-04-25T13:31:17.984 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:17 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:17.676+0000 7fa9e2c7e700 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:31:17.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:17 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:17.657+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:17.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:17 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:17.657+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:17.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:17 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:17.657+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:18.190 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:17 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:17.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:18.191 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:17 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:17.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:18.191 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:17 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:17.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:18.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:17 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:17.969+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:18.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:17 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:17.969+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:18.470 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:18 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:18.189+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:18.470 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:18 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:18.189+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:18.637 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:18 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:18.304+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:18.637 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:18 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:18.304+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:18.637 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:18 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:18.304+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.063774+0000 front 2024-04-25T13:30:47.063867+0000 (oldest deadline 2024-04-25T13:31:11.763703+0000) 2024-04-25T13:31:18.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:18 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:18.587+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:18.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:18 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:18.587+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:18.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:18 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:18.587+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:18.969 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:18 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:18.877+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:18.969 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:18 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:18.877+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:18.969 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:18 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:18.877+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:18.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:18 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:18.637+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:18.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:18 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:18.637+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:18.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:18 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:18.637+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:19.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:18 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:18.986+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:19.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:18 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:18.986+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:19.643 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:19 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:19.255+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:19.643 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:19 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:19.255+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:19.643 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:19 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:19.255+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.063774+0000 front 2024-04-25T13:30:47.063867+0000 (oldest deadline 2024-04-25T13:31:11.763703+0000) 2024-04-25T13:31:19.719 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:19 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:19.237+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:19.719 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:19 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:19.237+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:19.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:19 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:19.559+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:19.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:19 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:19.559+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:19.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:19 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:19.559+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:19.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:19 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:19.643+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:19.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:19 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:19.643+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:19.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:19 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:19.643+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:20.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:19 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:19.836+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:20.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:19 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:19.836+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:20.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:19 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:19.836+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:20.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:19 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:19.836+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:20.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:20 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:20.000+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:20.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:20 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:20.000+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:20.630 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:20 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:20.251+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:20.630 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:20 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:20.251+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:20.630 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:20 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:20.251+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.063774+0000 front 2024-04-25T13:30:47.063867+0000 (oldest deadline 2024-04-25T13:31:11.763703+0000) 2024-04-25T13:31:20.719 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:20 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:20.231+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:20.719 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:20 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:20.231+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:20.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:20 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:20.534+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:20.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:20 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:20.534+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:20.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:20 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:20.534+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:20.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:20 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:20.534+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:20.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:20 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:20.629+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:20.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:20 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:20.629+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:20.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:20 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:20.629+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:20.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:20 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:20.629+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:21.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:20 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:20.823+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:21.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:20 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:20.823+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:21.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:20 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:20.823+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:21.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:20 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:20.823+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:21.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:21 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:21.035+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:21.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:21 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:21.035+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:21.669 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:21 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:21.289+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:21.669 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:21 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:21.289+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:21.669 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:21 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:21.289+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.063774+0000 front 2024-04-25T13:30:47.063867+0000 (oldest deadline 2024-04-25T13:31:11.763703+0000) 2024-04-25T13:31:21.669 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:21 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:21.289+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:55.264772+0000 front 2024-04-25T13:30:55.264849+0000 (oldest deadline 2024-04-25T13:31:21.164911+0000) 2024-04-25T13:31:21.719 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:21 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:21.246+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:21.719 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:21 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:21.246+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:21.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:21 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:21.528+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:21.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:21 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:21.528+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:21.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:21 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:21.528+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:21.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:21 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:21.528+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:21.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:21 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:21.667+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:21.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:21 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:21.667+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:21.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:21 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:21.667+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:21.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:21 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:21.667+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:22.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:21 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:21.811+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:22.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:21 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:21.811+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:22.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:21 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:21.811+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:22.208 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:21 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:21.811+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:22.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:22 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:22.000+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:22.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:22 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:22.001+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:22.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:22 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:22.001+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.330765+0000 front 2024-04-25T13:30:58.330867+0000 (oldest deadline 2024-04-25T13:31:21.830765+0000) 2024-04-25T13:31:22.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:22 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:22.207+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:22.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:22 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:22.207+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:22.624 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:22 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:22.298+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.361039+0000 front 2024-04-25T13:30:34.361286+0000 (oldest deadline 2024-04-25T13:30:59.661055+0000) 2024-04-25T13:31:22.625 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:22 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:22.298+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.562195+0000 front 2024-04-25T13:30:42.562386+0000 (oldest deadline 2024-04-25T13:31:04.862182+0000) 2024-04-25T13:31:22.625 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:22 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:22.298+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.063774+0000 front 2024-04-25T13:30:47.063867+0000 (oldest deadline 2024-04-25T13:31:11.763703+0000) 2024-04-25T13:31:22.625 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:22 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6[57677]: 2024-04-25T13:31:22.298+0000 7fa9db26a700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:55.264772+0000 front 2024-04-25T13:30:55.264849+0000 (oldest deadline 2024-04-25T13:31:21.164911+0000) 2024-04-25T13:31:22.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:22 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:22.534+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:22.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:22 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:22.534+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:22.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:22 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:22.534+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:22.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:22 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:22.534+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:22.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:22 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:22.623+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:22.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:22 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:22.623+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:22.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:22 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:22.623+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:22.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:22 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:22.623+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:23.172 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:22 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:22.825+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:23.172 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:22 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:22.825+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:23.173 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:22 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:22.825+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:23.173 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:22 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:22.825+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:23.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:23 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:23.040+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:23.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:23 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:23.040+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:23.409 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:23 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:23.040+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.330765+0000 front 2024-04-25T13:30:58.330867+0000 (oldest deadline 2024-04-25T13:31:21.830765+0000) 2024-04-25T13:31:23.452 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:23 smithi031 podman[87219]: 2024-04-25 13:31:23.1750499 +0000 UTC m=+5.599998548 container died 15d2f28bd13cc8eaa56e540c276486a5441df6080c68a93aba60130bfaad0d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-25T13:31:23.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:23 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:23.171+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:23.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:23 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:23.171+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:23.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:23 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:23.171+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:23.734 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:23 smithi031 podman[87219]: 2024-04-25 13:31:23.551392761 +0000 UTC m=+5.976341405 container cleanup 15d2f28bd13cc8eaa56e540c276486a5441df6080c68a93aba60130bfaad0d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:31:23.734 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:23 smithi031 bash[87219]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6 2024-04-25T13:31:23.734 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:23 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:23.601+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:23.735 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:23 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:23.601+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:23.735 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:23 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:23.601+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:23.735 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:23 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:23.601+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:23.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:23 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:23.568+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:23.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:23 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:23.568+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:23.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:23 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:23.568+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:23.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:23 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:23.568+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:24.119 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:23 smithi031 podman[87238]: 2024-04-25 13:31:23.785789062 +0000 UTC m=+0.600422986 container remove 15d2f28bd13cc8eaa56e540c276486a5441df6080c68a93aba60130bfaad0d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:31:24.125 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:23 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:23.819+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:24.125 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:23 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:23.819+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:24.125 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:23 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:23.819+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:24.125 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:23 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:23.819+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:24.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:24 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:24.037+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:24.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:24 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:24.037+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:24.409 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:24 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:24.037+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.330765+0000 front 2024-04-25T13:30:58.330867+0000 (oldest deadline 2024-04-25T13:31:21.830765+0000) 2024-04-25T13:31:24.413 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:24 smithi031 podman[87335]: 2024-04-25 13:31:24.119935354 +0000 UTC m=+0.107742904 container create 14e6b3fda7773909f8fa8255a9ac8ab3418bb32372b27e5398a8c8fd5a5e4ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:31:24.413 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:24 smithi031 podman[87335]: 2024-04-25 13:31:24.031286988 +0000 UTC m=+0.019094502 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:31:24.414 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:24 smithi031 podman[87335]: 2024-04-25 13:31:24.339129272 +0000 UTC m=+0.326936780 container init 14e6b3fda7773909f8fa8255a9ac8ab3418bb32372b27e5398a8c8fd5a5e4ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:31:24.414 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:24 smithi031 podman[87335]: 2024-04-25 13:31:24.343117973 +0000 UTC m=+0.330925471 container start 14e6b3fda7773909f8fa8255a9ac8ab3418bb32372b27e5398a8c8fd5a5e4ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T13:31:24.414 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:24 smithi031 podman[87335]: 2024-04-25 13:31:24.413658484 +0000 UTC m=+0.401465994 container attach 14e6b3fda7773909f8fa8255a9ac8ab3418bb32372b27e5398a8c8fd5a5e4ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T13:31:24.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:24 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:24.124+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:24.470 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:24 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:24.124+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:24.470 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:24 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:24.124+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:24.734 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:24 smithi031 podman[87335]: 2024-04-25 13:31:24.643514868 +0000 UTC m=+0.631322377 container died 14e6b3fda7773909f8fa8255a9ac8ab3418bb32372b27e5398a8c8fd5a5e4ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:31:24.735 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:24 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:24.600+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:24.735 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:24 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:24.601+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:24.735 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:24 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:24.601+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:24.735 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:24 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:24.601+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:24.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:24 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:24.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:24.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:24 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:24.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:24.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:24 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:24.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:24.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:24 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:24.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:25.140 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:24 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:24.831+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:25.140 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:24 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:24.831+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:25.140 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:24 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:24.831+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:25.140 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:24 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:24.831+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:25.176 INFO:journalctl@ceph.osd.6.smithi031.stdout:Apr 25 13:31:25 smithi031 podman[87361]: 2024-04-25 13:31:25.167046416 +0000 UTC m=+0.518453026 container remove 14e6b3fda7773909f8fa8255a9ac8ab3418bb32372b27e5398a8c8fd5a5e4ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T13:31:25.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T13:31:25.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:31:25.222 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T13:31:25.222 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T13:31:25.222 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.7 2024-04-25T13:31:25.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:25.035+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:25.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:25.035+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:25.409 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:25.035+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.330765+0000 front 2024-04-25T13:30:58.330867+0000 (oldest deadline 2024-04-25T13:31:21.830765+0000) 2024-04-25T13:31:25.409 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:25 smithi066 systemd[1]: Stopping Ceph osd.7 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:31:25.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:25 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:25.139+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:25.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:25 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:25.139+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:25.470 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:25 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:25.139+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:25.908 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:25.522+0000 7f795a6ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:31:25.908 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:25.522+0000 7f795a6ef700 -1 osd.7 109 *** Got signal Terminated *** 2024-04-25T13:31:25.909 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:25.522+0000 7f795a6ef700 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:31:25.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:25.588+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:25.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:25.588+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:25.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:25.588+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:25.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:25 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:25.588+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:25.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:25 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:25.650+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:25.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:25 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:25.650+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:25.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:25 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:25.650+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:25.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:25 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:25.650+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:26.114 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:25 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:25.847+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:26.115 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:25 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:25.847+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:26.115 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:25 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:25.847+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:26.115 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:25 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:25.847+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:26.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:26 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:26.041+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:26.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:26 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:26.041+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:26.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:26 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:26.041+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.330765+0000 front 2024-04-25T13:30:58.330867+0000 (oldest deadline 2024-04-25T13:31:21.830765+0000) 2024-04-25T13:31:26.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:26 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:26.114+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:26.470 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:26 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:26.114+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:26.470 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:26 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:26.114+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:26.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:26 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:26.576+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:26.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:26 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:26.576+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:26.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:26 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:26.576+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:26.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:26 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:26.576+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:26.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:26 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:26.687+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:26.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:26 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:26.687+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:26.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:26 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:26.687+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:26.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:26 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:26.687+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:27.159 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:26 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:26.846+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:27.160 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:26 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:26.846+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:27.160 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:26 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:26.846+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:27.160 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:26 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:26.846+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:27.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:26 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:26.996+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:27.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:26 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:26.996+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:27.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:26 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:26.996+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.330765+0000 front 2024-04-25T13:30:58.330867+0000 (oldest deadline 2024-04-25T13:31:21.830765+0000) 2024-04-25T13:31:27.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:27 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:27.158+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:27.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:27 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:27.158+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:27.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:27 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:27.158+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:27.700 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:27 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:27.625+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:27.700 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:27 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:27.625+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:27.700 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:27 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:27.625+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:27.700 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:27 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:27.625+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:28.146 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:27 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:27.803+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:28.146 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:27 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:27.803+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:28.147 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:27 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:27.803+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:28.147 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:27 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:27.803+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:28.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:27 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:27.736+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:28.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:27 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:27.736+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:28.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:27 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:27.736+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:28.235 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:27 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:27.736+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:28.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:28.030+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:28.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:28.030+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:28.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:28.030+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.330765+0000 front 2024-04-25T13:30:58.330867+0000 (oldest deadline 2024-04-25T13:31:21.830765+0000) 2024-04-25T13:31:28.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:28.146+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:28.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:28.146+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:28.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:28.146+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:28.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:28.601+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:28.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:28.601+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:28.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:28.601+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:28.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:28.601+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:29.168 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:28.787+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:29.169 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:28.787+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:29.169 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:28.787+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:29.169 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:28.787+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:29.169 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:28 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:28.787+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:29.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:28.780+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:29.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:28.780+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:29.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:28.780+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:29.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:28.780+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:29.235 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:28 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:28.780+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:29.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:29 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:29.050+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:29.409 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:29 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:29.050+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:29.409 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:29 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:29.050+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.330765+0000 front 2024-04-25T13:30:58.330867+0000 (oldest deadline 2024-04-25T13:31:21.830765+0000) 2024-04-25T13:31:29.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:29 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:29.168+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:29.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:29 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:29.168+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:29.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:29 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:29.168+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:29.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:29 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:29.643+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:29.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:29 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:29.643+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:29.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:29 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:29.643+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:29.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:29 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:29.643+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:29.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:29 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:29.731+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:29.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:29 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:29.731+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:29.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:29 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:29.731+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:29.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:29 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:29.731+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:29.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:29 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:29.731+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:30.156 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:29 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:29.790+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:30.156 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:29 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:29.790+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:30.156 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:29 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:29.790+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:30.156 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:29 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:29.790+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:30.156 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:29 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:29.790+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:30.408 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:30.071+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:42.627619+0000 front 2024-04-25T13:30:42.627660+0000 (oldest deadline 2024-04-25T13:31:05.527708+0000) 2024-04-25T13:31:30.409 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:30.071+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:48.428918+0000 front 2024-04-25T13:30:48.428833+0000 (oldest deadline 2024-04-25T13:31:13.729007+0000) 2024-04-25T13:31:30.409 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:30.071+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.330765+0000 front 2024-04-25T13:30:58.330867+0000 (oldest deadline 2024-04-25T13:31:21.830765+0000) 2024-04-25T13:31:30.409 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7[51368]: 2024-04-25T13:31:30.071+0000 7f7956ce3700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.731704+0000 front 2024-04-25T13:31:04.731743+0000 (oldest deadline 2024-04-25T13:31:29.431816+0000) 2024-04-25T13:31:30.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:30.155+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:30.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:30.155+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:30.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:30.155+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:30.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:30.632+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:30.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:30.632+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:30.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:30.632+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:30.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:30 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:30.632+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:30.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:30.710+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:30.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:30.710+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:30.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:30.710+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:30.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:30.710+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:30.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:30 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:30.710+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:31.139 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:30.831+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:31.139 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:30.831+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:31.139 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:30.831+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:31.139 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:30.831+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:31.139 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:30 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:30.831+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:31.298 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:31 smithi066 podman[69674]: 2024-04-25 13:31:31.000296963 +0000 UTC m=+5.579918247 container died 1d2bc6b414c5390bdb67ddfdad31842c4ef8fd5a5e94977c4cd09c236dfc9481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:31:31.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:31 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:31.138+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:31.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:31 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:31.138+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:31.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:31 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:31.138+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:31.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:31 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:31.138+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:06.055668+0000 front 2024-04-25T13:31:06.055755+0000 (oldest deadline 2024-04-25T13:31:30.755788+0000) 2024-04-25T13:31:31.590 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:31 smithi066 podman[69674]: 2024-04-25 13:31:31.397613241 +0000 UTC m=+5.977234526 container cleanup 1d2bc6b414c5390bdb67ddfdad31842c4ef8fd5a5e94977c4cd09c236dfc9481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:31:31.590 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:31 smithi066 bash[69674]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7 2024-04-25T13:31:31.908 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:31 smithi066 podman[69703]: 2024-04-25 13:31:31.590381842 +0000 UTC m=+0.588925980 container remove 1d2bc6b414c5390bdb67ddfdad31842c4ef8fd5a5e94977c4cd09c236dfc9481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T13:31:31.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:31 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:31.596+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:31.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:31 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:31.596+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:31.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:31 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:31.597+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:31.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:31 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:31.597+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:31.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:31 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:31.597+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:31.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:31 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:31.710+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:31.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:31 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:31.710+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:31.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:31 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:31.710+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:31.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:31 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:31.710+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:31.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:31 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:31.710+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:32.107 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:31 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:31.838+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:32.107 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:31 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:31.838+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:32.108 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:31 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:31.838+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:32.108 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:31 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:31.838+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:32.108 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:31 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:31.838+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:32.302 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:32 smithi066 podman[69793]: 2024-04-25 13:31:32.024622969 +0000 UTC m=+0.114020492 container create 44738c2d7b86f82a345a37e62648cd0ea629d5f7287e046bc0960aec221665f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T13:31:32.302 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:32 smithi066 podman[69793]: 2024-04-25 13:31:31.9292928 +0000 UTC m=+0.018690352 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:31:32.302 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:32 smithi066 podman[69793]: 2024-04-25 13:31:32.218984005 +0000 UTC m=+0.308381531 container init 44738c2d7b86f82a345a37e62648cd0ea629d5f7287e046bc0960aec221665f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T13:31:32.302 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:32 smithi066 podman[69793]: 2024-04-25 13:31:32.222319941 +0000 UTC m=+0.311717465 container start 44738c2d7b86f82a345a37e62648cd0ea629d5f7287e046bc0960aec221665f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T13:31:32.302 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:32 smithi066 podman[69793]: 2024-04-25 13:31:32.301924868 +0000 UTC m=+0.391322392 container attach 44738c2d7b86f82a345a37e62648cd0ea629d5f7287e046bc0960aec221665f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T13:31:32.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:32.107+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:32.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:32.107+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:32.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:32.107+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:32.470 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:32.107+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:06.055668+0000 front 2024-04-25T13:31:06.055755+0000 (oldest deadline 2024-04-25T13:31:30.755788+0000) 2024-04-25T13:31:32.597 INFO:journalctl@ceph.osd.7.smithi066.stdout:Apr 25 13:31:32 smithi066 podman[69793]: 2024-04-25 13:31:32.527356309 +0000 UTC m=+0.616753838 container died 44738c2d7b86f82a345a37e62648cd0ea629d5f7287e046bc0960aec221665f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T13:31:32.871 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:32 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:32.596+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:32.871 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:32 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:32.596+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:32.871 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:32 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:32.596+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:32.872 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:32 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:32.596+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:32.872 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:32 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:32.596+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:32.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:32 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:32.694+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:32.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:32 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:32.694+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:32.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:32 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:32.694+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:32.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:32 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:32.694+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:32.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:32 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:32.694+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:33.196 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T13:31:33.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:33.061+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:33.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:33.061+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:33.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:33.061+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:33.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:33.061+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:06.055668+0000 front 2024-04-25T13:31:06.055755+0000 (oldest deadline 2024-04-25T13:31:30.755788+0000) 2024-04-25T13:31:33.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:32.812+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:33.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:32.812+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:33.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:32.812+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:33.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:32.812+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:33.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:32 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:32.812+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:33.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:31:33.240 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T13:31:33.240 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T13:31:33.241 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.8 2024-04-25T13:31:33.525 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:33 smithi136 systemd[1]: Stopping Ceph osd.8 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:31:33.831 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:33.524+0000 7f6ca8e06700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:31:33.831 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:33.524+0000 7f6ca8e06700 -1 osd.8 109 *** Got signal Terminated *** 2024-04-25T13:31:33.832 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:33.524+0000 7f6ca8e06700 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:31:33.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:33.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:33.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:33.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:33.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:33.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:33.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:33.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:33.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:33.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:33.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:33 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:33.675+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:33.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:33 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:33.675+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:33.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:33 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:33.675+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:33.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:33 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:33.675+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:33.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:33 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:33.675+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:34.221 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:34 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:34.024+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:34.221 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:34 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:34.024+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:34.221 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:34 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:34.024+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:34.221 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:34 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:34.024+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:06.055668+0000 front 2024-04-25T13:31:06.055755+0000 (oldest deadline 2024-04-25T13:31:30.755788+0000) 2024-04-25T13:31:34.221 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:33.830+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:34.222 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:33.830+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:34.222 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:33.830+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:34.222 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:33.830+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:34.222 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:33 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:33.830+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:34.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:34 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:34.529+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:34.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:34 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:34.529+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:34.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:34 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:34.529+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:34.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:34 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:34.529+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:34.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:34 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:34.529+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:34.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:34 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:34.661+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:34.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:34 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:34.661+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:34.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:34 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:34.661+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:34.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:34 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:34.661+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:34.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:34 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:34.661+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:35.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:35 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:35.008+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:35.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:35 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:35.008+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:35.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:35 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:35.008+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:35.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:35 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:35.008+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:06.055668+0000 front 2024-04-25T13:31:06.055755+0000 (oldest deadline 2024-04-25T13:31:30.755788+0000) 2024-04-25T13:31:35.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:34 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:34.815+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:35.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:34 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:34.815+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:35.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:34 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:34.815+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:35.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:34 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:34.815+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:35.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:34 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:34.815+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:35.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:35 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:35.502+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:35.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:35 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:35.502+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:35.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:35 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:35.502+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:35.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:35 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:35.502+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:35.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:35 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:35.502+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:35.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:35.700+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:35.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:35.700+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:35.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:35.700+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:35.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:35.700+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:35.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:35 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:35.700+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:36.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:36 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:36.029+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:36.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:36 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:36.029+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:36.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:36 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:36.029+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:36.220 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:36 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:36.029+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:06.055668+0000 front 2024-04-25T13:31:06.055755+0000 (oldest deadline 2024-04-25T13:31:30.755788+0000) 2024-04-25T13:31:36.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:35 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:35.838+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:36.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:35 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:35.838+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:36.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:35 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:35.838+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:36.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:35 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:35.838+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:36.221 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:35 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:35.838+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:36.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:36 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:36.475+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:36.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:36 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:36.475+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:36.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:36 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:36.475+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:36.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:36 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:36.475+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:36.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:36 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:36.475+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:36.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:36 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:36.689+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:36.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:36 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:36.689+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:36.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:36 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:36.689+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:36.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:36 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:36.689+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:36.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:36 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:36.689+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:37.121 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:37 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:37.060+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:37.121 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:37 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:37.060+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:37.121 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:37 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:37.060+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:37.121 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:37 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:37.060+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:06.055668+0000 front 2024-04-25T13:31:06.055755+0000 (oldest deadline 2024-04-25T13:31:30.755788+0000) 2024-04-25T13:31:37.121 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:36 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:36.862+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:37.122 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:36 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:36.862+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:37.122 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:36 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:36.862+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:37.122 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:36 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:36.862+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:37.122 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:36 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:36.862+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:37.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:37 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:37.497+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:37.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:37 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:37.497+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:37.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:37 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:37.497+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:37.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:37 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:37.497+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:37.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:37 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:37.497+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:37.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:37 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:37.730+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:37.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:37 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:37.730+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:37.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:37 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:37.730+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:37.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:37 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:37.730+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:37.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:37 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:37.730+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:38.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:38 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:38.082+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:38.651804+0000 front 2024-04-25T13:30:38.651788+0000 (oldest deadline 2024-04-25T13:31:07.452269+0000) 2024-04-25T13:31:38.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:38 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:38.082+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:47.452637+0000 front 2024-04-25T13:30:47.452736+0000 (oldest deadline 2024-04-25T13:31:11.552858+0000) 2024-04-25T13:31:38.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:38 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:38.082+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:57.954581+0000 front 2024-04-25T13:30:57.954341+0000 (oldest deadline 2024-04-25T13:31:22.654293+0000) 2024-04-25T13:31:38.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:38 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8[51458]: 2024-04-25T13:31:38.082+0000 7f6ca13f2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:06.055668+0000 front 2024-04-25T13:31:06.055755+0000 (oldest deadline 2024-04-25T13:31:30.755788+0000) 2024-04-25T13:31:38.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:37 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:37.889+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:38.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:37 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:37.889+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:38.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:37 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:37.889+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:38.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:37 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:37.889+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:38.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:37 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:37.889+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:38.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:38 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:38.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:38.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:38 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:38.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:38.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:38 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:38.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:38.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:38 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:38.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:38.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:38 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:38.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:38.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:38 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:38.712+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:38.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:38 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:38.712+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:38.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:38 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:38.712+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:38.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:38 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:38.712+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:38.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:38 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:38.712+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:39.219 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:38 smithi136 podman[69623]: 2024-04-25 13:31:38.968737295 +0000 UTC m=+5.537457341 container died 2855554da34a9a93ec2ab8f67bc7f3857ee0d0ea3ff32b24c7c6fa442e179c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:31:39.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:38 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:38.861+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:39.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:38 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:38.861+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:39.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:38 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:38.861+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:39.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:38 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:38.861+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:39.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:38 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:38.861+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:39.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:38 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:38.861+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:39.592 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:39 smithi136 podman[69623]: 2024-04-25 13:31:39.324860947 +0000 UTC m=+5.893580980 container cleanup 2855554da34a9a93ec2ab8f67bc7f3857ee0d0ea3ff32b24c7c6fa442e179c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T13:31:39.593 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:39 smithi136 bash[69623]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8 2024-04-25T13:31:39.861 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:39 smithi136 podman[69653]: 2024-04-25 13:31:39.592624532 +0000 UTC m=+0.623222078 container remove 2855554da34a9a93ec2ab8f67bc7f3857ee0d0ea3ff32b24c7c6fa442e179c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T13:31:39.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:39 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:39.548+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:39.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:39 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:39.548+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:39.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:39 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:39.548+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:39.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:39 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:39.548+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:39.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:39 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:39.548+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:39.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:39 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:39.548+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:40.114 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:39 smithi136 podman[69743]: 2024-04-25 13:31:39.843477346 +0000 UTC m=+0.018069227 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:31:40.114 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:39 smithi136 podman[69743]: 2024-04-25 13:31:39.985113554 +0000 UTC m=+0.159705421 container create 6e1a77ad841c2f66c5bf94ff0a07549054746fca028cbee2e6246a09419fe34f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T13:31:40.114 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:39 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:39.860+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:40.114 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:39 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:39.860+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:40.114 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:39 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:39.860+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:40.115 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:39 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:39.860+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:40.115 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:39 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:39.860+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:40.115 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:39 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:39.860+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:40.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:39 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:39.760+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:40.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:39 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:39.760+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:40.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:39 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:39.760+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:40.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:39 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:39.760+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:40.235 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:39 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:39.760+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:40.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:40 smithi136 podman[69743]: 2024-04-25 13:31:40.212301863 +0000 UTC m=+0.386893731 container init 6e1a77ad841c2f66c5bf94ff0a07549054746fca028cbee2e6246a09419fe34f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:31:40.470 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:40 smithi136 podman[69743]: 2024-04-25 13:31:40.215383136 +0000 UTC m=+0.389975005 container start 6e1a77ad841c2f66c5bf94ff0a07549054746fca028cbee2e6246a09419fe34f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T13:31:40.470 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:40 smithi136 podman[69743]: 2024-04-25 13:31:40.295568463 +0000 UTC m=+0.470160331 container attach 6e1a77ad841c2f66c5bf94ff0a07549054746fca028cbee2e6246a09419fe34f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:31:40.790 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 13:31:40 smithi136 podman[69743]: 2024-04-25 13:31:40.507521186 +0000 UTC m=+0.682113048 container died 6e1a77ad841c2f66c5bf94ff0a07549054746fca028cbee2e6246a09419fe34f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T13:31:40.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:40 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:40.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:40.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:40 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:40.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:40.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:40 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:40.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:40.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:40 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:40.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:40.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:40 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:40.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:40.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:40 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:40.557+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:41.115 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:40 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:40.880+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:41.115 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:40 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:40.880+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:41.116 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:40 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:40.880+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:41.116 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:40 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:40.880+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:41.116 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:40 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:40.880+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:41.116 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:40 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:40.880+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:41.137 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T13:31:41.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:31:41.181 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T13:31:41.181 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T13:31:41.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.9 2024-04-25T13:31:41.208 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:40 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:40.755+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:41.209 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:40 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:40.755+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:41.209 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:40 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:40.755+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:41.209 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:40 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:40.755+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:41.209 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:40 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:40.755+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:41.484 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:41 smithi031 systemd[1]: Stopping Ceph osd.9 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:31:41.566 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:41 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:41.450+0000 7fbb9b923700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:31:41.566 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:41 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:41.450+0000 7fbb9b923700 -1 osd.9 109 *** Got signal Terminated *** 2024-04-25T13:31:41.566 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:41 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:41.450+0000 7fbb9b923700 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:31:41.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:41 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:41.583+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:41.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:41 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:41.583+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:41.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:41 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:41.583+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:41.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:41 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:41.583+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:41.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:41 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:41.583+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:41.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:41 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:41.583+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:41.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:41 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:41.803+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:41.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:41 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:41.803+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:41.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:41 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:41.803+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:41.984 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:41 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:41.803+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:41.985 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:41 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:41.803+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:42.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:41 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:41.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:42.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:41 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:41.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:42.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:41 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:41.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:42.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:41 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:41.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:42.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:41 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:41.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:42.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:41 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:41.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:42.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:42 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:42.555+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:42.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:42 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:42.555+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:42.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:42 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:42.555+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:42.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:42 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:42.555+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:42.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:42 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:42.555+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:42.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:42 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:42.555+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:43.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:42 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:42.873+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:43.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:42 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:42.873+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:43.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:42 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:42.873+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:43.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:42 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:42.873+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:43.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:42 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:42.873+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:43.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:42 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:42.873+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:43.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:42 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:42.800+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:43.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:42 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:42.800+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:43.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:42 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:42.800+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:43.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:42 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:42.800+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:43.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:42 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:42.800+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:43.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:43 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:43.582+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:43.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:43 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:43.582+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:43.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:43 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:43.582+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:43.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:43 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:43.582+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:43.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:43 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:43.582+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:43.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:43 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:43.582+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:44.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:43 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:43.892+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:44.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:43 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:43.892+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:44.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:43 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:43.892+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:44.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:43 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:43.892+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:44.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:43 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:43.892+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:44.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:43 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:43.892+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:44.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:43 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:43.771+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:44.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:43 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:43.771+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:44.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:43 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:43.771+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:44.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:43 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:43.771+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:44.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:43 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:43.771+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:44.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:44 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:44.569+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:44.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:44 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:44.569+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:44.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:44 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:44.569+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:44.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:44 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:44.569+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:44.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:44 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:44.569+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:44.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:44 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:44.569+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:45.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:44 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:44.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:45.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:44 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:44.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:45.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:44 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:44.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:45.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:44 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:44.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:45.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:44 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:44.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:45.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:44 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:44.878+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:45.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:44 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:44.751+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:45.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:44 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:44.751+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:45.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:44 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:44.751+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:45.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:44 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:44.751+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:45.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:44 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:44.751+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:45.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:45 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:45.565+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:45.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:45 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:45.565+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:45.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:45 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:45.565+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:45.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:45 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:45.565+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:45.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:45 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:45.565+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:45.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:45 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:45.565+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:45.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:45 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:45.565+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T13:31:19.197124+0000 front 2024-04-25T13:31:19.197185+0000 (oldest deadline 2024-04-25T13:31:45.097009+0000) 2024-04-25T13:31:46.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:45.888+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:46.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:45.888+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:46.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:45.888+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:46.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:45.888+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:46.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:45.888+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:46.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:45 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:45.888+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:46.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:45.795+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:34.805609+0000 front 2024-04-25T13:30:34.805795+0000 (oldest deadline 2024-04-25T13:30:56.505734+0000) 2024-04-25T13:31:46.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:45.795+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.207058+0000 front 2024-04-25T13:30:41.207205+0000 (oldest deadline 2024-04-25T13:31:07.607426+0000) 2024-04-25T13:31:46.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:45.795+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.508643+0000 front 2024-04-25T13:30:50.508682+0000 (oldest deadline 2024-04-25T13:31:14.008465+0000) 2024-04-25T13:31:46.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:45.795+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.008889+0000 front 2024-04-25T13:30:54.008778+0000 (oldest deadline 2024-04-25T13:31:19.908920+0000) 2024-04-25T13:31:46.235 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:45.795+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.010072+0000 front 2024-04-25T13:31:04.009934+0000 (oldest deadline 2024-04-25T13:31:28.110080+0000) 2024-04-25T13:31:46.235 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:45 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9[64704]: 2024-04-25T13:31:45.795+0000 7fbb93f0f700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T13:31:20.714273+0000 front 2024-04-25T13:31:20.714241+0000 (oldest deadline 2024-04-25T13:31:45.414512+0000) 2024-04-25T13:31:46.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:46 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:46.568+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:46.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:46 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:46.568+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:46.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:46 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:46.568+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:46.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:46 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:46.568+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:46.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:46 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:46.568+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:46.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:46 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:46.568+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:46.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:46 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:46.568+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T13:31:19.197124+0000 front 2024-04-25T13:31:19.197185+0000 (oldest deadline 2024-04-25T13:31:45.097009+0000) 2024-04-25T13:31:47.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:46 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:46.924+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:47.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:46 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:46.924+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:47.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:46 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:46.924+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:47.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:46 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:46.924+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:47.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:46 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:46.924+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:47.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:46 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:46.924+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:47.234 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:46 smithi031 podman[87453]: 2024-04-25 13:31:46.907880672 +0000 UTC m=+5.527407497 container died 918cc8e8e56fc93503681b4732c09d92cf4daaf00329e7e64cc0155720295aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T13:31:47.559 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:47 smithi031 podman[87453]: 2024-04-25 13:31:47.309006512 +0000 UTC m=+5.928533346 container cleanup 918cc8e8e56fc93503681b4732c09d92cf4daaf00329e7e64cc0155720295aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T13:31:47.560 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:47 smithi031 bash[87453]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9 2024-04-25T13:31:47.877 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:47 smithi031 podman[87471]: 2024-04-25 13:31:47.559838224 +0000 UTC m=+0.649324745 container remove 918cc8e8e56fc93503681b4732c09d92cf4daaf00329e7e64cc0155720295aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T13:31:47.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:47 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:47.526+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:47.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:47 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:47.526+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:47.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:47 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:47.526+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:47.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:47 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:47.526+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:47.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:47 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:47.526+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:47.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:47 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:47.526+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:47.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:47 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:47.526+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T13:31:19.197124+0000 front 2024-04-25T13:31:19.197185+0000 (oldest deadline 2024-04-25T13:31:45.097009+0000) 2024-04-25T13:31:48.139 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:47 smithi031 podman[87561]: 2024-04-25 13:31:47.780195028 +0000 UTC m=+0.018556280 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:31:48.139 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:48 smithi031 podman[87561]: 2024-04-25 13:31:48.019082281 +0000 UTC m=+0.257443546 container create 9a00c180bebe8a9b281bd3074db50e8de83401f547a297757146abfd9f3c193d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:31:48.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:47 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:47.941+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:48.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:47 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:47.941+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:48.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:47 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:47.941+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:48.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:47 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:47.941+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:48.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:47 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:47.941+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:48.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:47 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:47.941+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:48.484 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:48 smithi031 podman[87561]: 2024-04-25 13:31:48.246378605 +0000 UTC m=+0.484739827 container init 9a00c180bebe8a9b281bd3074db50e8de83401f547a297757146abfd9f3c193d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T13:31:48.484 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:48 smithi031 podman[87561]: 2024-04-25 13:31:48.249546477 +0000 UTC m=+0.487907700 container start 9a00c180bebe8a9b281bd3074db50e8de83401f547a297757146abfd9f3c193d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T13:31:48.485 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:48 smithi031 podman[87561]: 2024-04-25 13:31:48.321269842 +0000 UTC m=+0.559631080 container attach 9a00c180bebe8a9b281bd3074db50e8de83401f547a297757146abfd9f3c193d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T13:31:48.883 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:48 smithi031 podman[87561]: 2024-04-25 13:31:48.559623827 +0000 UTC m=+0.797985055 container died 9a00c180bebe8a9b281bd3074db50e8de83401f547a297757146abfd9f3c193d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T13:31:48.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:48 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:48.545+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:48.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:48 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:48.545+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:48.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:48 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:48.545+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:48.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:48 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:48.545+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:48.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:48 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:48.545+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:48.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:48 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:48.545+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:48.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:48 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:48.545+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T13:31:19.197124+0000 front 2024-04-25T13:31:19.197185+0000 (oldest deadline 2024-04-25T13:31:45.097009+0000) 2024-04-25T13:31:49.134 INFO:journalctl@ceph.osd.9.smithi031.stdout:Apr 25 13:31:49 smithi031 podman[87587]: 2024-04-25 13:31:49.115718837 +0000 UTC m=+0.545331710 container remove 9a00c180bebe8a9b281bd3074db50e8de83401f547a297757146abfd9f3c193d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:31:49.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T13:31:49.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:48 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:48.893+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:49.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:48 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:48.893+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:49.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:48 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:48.893+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:49.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:48 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:48.893+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:49.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:48 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:48.893+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:49.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:48 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:48.893+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:49.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:31:49.222 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T13:31:49.222 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T13:31:49.222 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.10 2024-04-25T13:31:49.525 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:49 smithi066 systemd[1]: Stopping Ceph osd.10 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:31:49.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:49 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:49.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:49.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:49 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:49.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:49.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:49 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:49.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:49.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:49 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:49.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:49.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:49 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:49.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:49.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:49 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:49.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:49.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:49 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:49.525+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T13:31:19.197124+0000 front 2024-04-25T13:31:19.197185+0000 (oldest deadline 2024-04-25T13:31:45.097009+0000) 2024-04-25T13:31:49.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:49 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:49.531+0000 7ff0f9252700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:31:49.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:49 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:49.531+0000 7ff0f9252700 -1 osd.10 109 *** Got signal Terminated *** 2024-04-25T13:31:49.911 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:49 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:49.531+0000 7ff0f9252700 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:31:50.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:49 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:49.932+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:50.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:49 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:49.932+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:50.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:49 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:49.932+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:50.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:49 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:49.932+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:50.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:49 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:49.932+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:50.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:49 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:49.932+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:50.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:50.493+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:50.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:50.493+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:50.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:50.493+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:50.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:50.493+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:50.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:50.493+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:50.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:50.493+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:50.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:50 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:50.493+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T13:31:19.197124+0000 front 2024-04-25T13:31:19.197185+0000 (oldest deadline 2024-04-25T13:31:45.097009+0000) 2024-04-25T13:31:51.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:50.945+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:51.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:50.945+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:51.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:50.945+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:51.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:50.945+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:51.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:50.945+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:51.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:50 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:50.945+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:51.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:51 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:51.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:51.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:51 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:51.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:51.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:51 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:51.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:51.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:51 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:51.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:51.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:51 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:51.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:51.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:51 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:51.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:51.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:51 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:51.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T13:31:19.197124+0000 front 2024-04-25T13:31:19.197185+0000 (oldest deadline 2024-04-25T13:31:45.097009+0000) 2024-04-25T13:31:52.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:51.906+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:52.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:51.906+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:52.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:51.906+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:52.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:51.906+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:52.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:51.906+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:52.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:51.906+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:52.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:52 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:52.564+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:52.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:52 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:52.564+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:52.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:52 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:52.564+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:52.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:52 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:52.564+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:52.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:52 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:52.564+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:52.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:52 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:52.564+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:52.910 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:52 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:52.564+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T13:31:19.197124+0000 front 2024-04-25T13:31:19.197185+0000 (oldest deadline 2024-04-25T13:31:45.097009+0000) 2024-04-25T13:31:53.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:52 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:52.923+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:53.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:52 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:52.923+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:53.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:52 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:52.923+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:53.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:52 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:52.923+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:53.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:52 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:52.923+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:53.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:52 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:52.923+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:53.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:53.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:53.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:53.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:53.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:53.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:53.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:53.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:53.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:53.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:53.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:53.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:53.909 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:53 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:53.517+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T13:31:19.197124+0000 front 2024-04-25T13:31:19.197185+0000 (oldest deadline 2024-04-25T13:31:45.097009+0000) 2024-04-25T13:31:54.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:53 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:53.936+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:54.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:53 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:53.936+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:54.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:53 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:53.936+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:54.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:53 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:53.936+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:54.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:53 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:53.936+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:54.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:53 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:53.936+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:54.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:53 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:53.936+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-04-25T13:31:29.002634+0000 front 2024-04-25T13:31:29.002587+0000 (oldest deadline 2024-04-25T13:31:53.702256+0000) 2024-04-25T13:31:54.843 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:54 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:54.506+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:30.388868+0000 front 2024-04-25T13:30:30.388780+0000 (oldest deadline 2024-04-25T13:30:55.688976+0000) 2024-04-25T13:31:54.844 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:54 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:54.506+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:40.290523+0000 front 2024-04-25T13:30:40.290493+0000 (oldest deadline 2024-04-25T13:31:06.190663+0000) 2024-04-25T13:31:54.844 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:54 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:54.506+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.792610+0000 front 2024-04-25T13:30:50.792466+0000 (oldest deadline 2024-04-25T13:31:14.292445+0000) 2024-04-25T13:31:54.844 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:54 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:54.506+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:54.293035+0000 front 2024-04-25T13:30:54.293175+0000 (oldest deadline 2024-04-25T13:31:19.592949+0000) 2024-04-25T13:31:54.844 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:54 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:54.506+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:04.194523+0000 front 2024-04-25T13:31:04.194573+0000 (oldest deadline 2024-04-25T13:31:31.194953+0000) 2024-04-25T13:31:54.844 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:54 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:54.506+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.896952+0000 front 2024-04-25T13:31:13.896892+0000 (oldest deadline 2024-04-25T13:31:39.196516+0000) 2024-04-25T13:31:54.844 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:54 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10[58420]: 2024-04-25T13:31:54.506+0000 7ff0f183e700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-04-25T13:31:19.197124+0000 front 2024-04-25T13:31:19.197185+0000 (oldest deadline 2024-04-25T13:31:45.097009+0000) 2024-04-25T13:31:55.158 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:54 smithi066 podman[69900]: 2024-04-25 13:31:54.843573635 +0000 UTC m=+5.417444267 container died 32c4c522f4bf2b8cf2737edc2679383bcf93f222f32465725ae2fa1023d965d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:31:55.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:54 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:54.961+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:55.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:54 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:54.961+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:55.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:54 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:54.961+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:55.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:54 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:54.961+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:55.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:54 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:54.961+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:55.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:54 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:54.961+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:55.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:54 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:54.961+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-04-25T13:31:29.002634+0000 front 2024-04-25T13:31:29.002587+0000 (oldest deadline 2024-04-25T13:31:53.702256+0000) 2024-04-25T13:31:55.566 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:55 smithi066 podman[69900]: 2024-04-25 13:31:55.315252577 +0000 UTC m=+5.889123209 container cleanup 32c4c522f4bf2b8cf2737edc2679383bcf93f222f32465725ae2fa1023d965d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T13:31:55.566 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:55 smithi066 bash[69900]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10 2024-04-25T13:31:55.908 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:55 smithi066 podman[69917]: 2024-04-25 13:31:55.566580459 +0000 UTC m=+0.718758526 container remove 32c4c522f4bf2b8cf2737edc2679383bcf93f222f32465725ae2fa1023d965d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:31:56.253 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:55 smithi066 podman[70011]: 2024-04-25 13:31:55.840048011 +0000 UTC m=+0.018279450 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:31:56.253 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:55 smithi066 podman[70011]: 2024-04-25 13:31:55.94262157 +0000 UTC m=+0.120853041 container create 8b310c164b71b6c93049c5fa3f45a7d3bc765993b5a4c0390605916d9caa601f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:31:56.253 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:56 smithi066 podman[70011]: 2024-04-25 13:31:56.170292478 +0000 UTC m=+0.348523918 container init 8b310c164b71b6c93049c5fa3f45a7d3bc765993b5a4c0390605916d9caa601f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T13:31:56.253 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:56 smithi066 podman[70011]: 2024-04-25 13:31:56.174129863 +0000 UTC m=+0.352361299 container start 8b310c164b71b6c93049c5fa3f45a7d3bc765993b5a4c0390605916d9caa601f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:31:56.254 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:56 smithi066 podman[70011]: 2024-04-25 13:31:56.253152285 +0000 UTC m=+0.431383724 container attach 8b310c164b71b6c93049c5fa3f45a7d3bc765993b5a4c0390605916d9caa601f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T13:31:56.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.005+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:56.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.005+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:56.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.005+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:56.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.005+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:56.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.005+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:56.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.005+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:56.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.005+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-04-25T13:31:29.002634+0000 front 2024-04-25T13:31:29.002587+0000 (oldest deadline 2024-04-25T13:31:53.702256+0000) 2024-04-25T13:31:56.658 INFO:journalctl@ceph.osd.10.smithi066.stdout:Apr 25 13:31:56 smithi066 podman[70011]: 2024-04-25 13:31:56.482293635 +0000 UTC m=+0.660525077 container died 8b310c164b71b6c93049c5fa3f45a7d3bc765993b5a4c0390605916d9caa601f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:31:57.032 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T13:31:57.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:31:57.071 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T13:31:57.071 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T13:31:57.071 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.11 2024-04-25T13:31:57.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.959+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:57.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.959+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:57.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.959+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:57.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.959+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:57.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.959+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:57.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.959+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:57.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:56.959+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-04-25T13:31:29.002634+0000 front 2024-04-25T13:31:29.002587+0000 (oldest deadline 2024-04-25T13:31:53.702256+0000) 2024-04-25T13:31:57.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:57 smithi136 systemd[1]: Stopping Ceph osd.11 for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:31:57.720 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:57.374+0000 7f96ec0ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:31:57.720 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:57.374+0000 7f96ec0ed700 -1 osd.11 109 *** Got signal Terminated *** 2024-04-25T13:31:57.720 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:57.374+0000 7f96ec0ed700 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:31:58.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:57.998+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:58.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:57.998+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:58.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:57.998+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:58.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:57.998+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:58.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:57.998+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:58.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:57.998+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:58.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:57 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:57.998+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-04-25T13:31:29.002634+0000 front 2024-04-25T13:31:29.002587+0000 (oldest deadline 2024-04-25T13:31:53.702256+0000) 2024-04-25T13:31:59.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:58 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:58.948+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:31:59.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:58 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:58.948+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:31:59.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:58 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:58.948+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:31:59.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:58 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:58.948+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:31:59.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:58 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:58.948+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:31:59.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:58 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:58.948+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:31:59.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:58 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:58.948+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-04-25T13:31:29.002634+0000 front 2024-04-25T13:31:29.002587+0000 (oldest deadline 2024-04-25T13:31:53.702256+0000) 2024-04-25T13:32:00.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:59 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:59.977+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:32:00.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:59 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:59.977+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:32:00.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:59 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:59.977+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:32:00.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:59 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:59.977+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:32:00.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:59 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:59.977+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:32:00.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:59 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:59.977+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:32:00.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:31:59 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:31:59.977+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-04-25T13:31:29.002634+0000 front 2024-04-25T13:31:29.002587+0000 (oldest deadline 2024-04-25T13:31:53.702256+0000) 2024-04-25T13:32:01.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:00 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:00.949+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:32:01.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:00 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:00.949+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:32:01.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:00 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:00.949+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:32:01.219 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:00 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:00.949+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:32:01.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:00 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:00.949+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:32:01.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:00 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:00.949+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:32:01.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:00 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:00.949+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-04-25T13:31:29.002634+0000 front 2024-04-25T13:31:29.002587+0000 (oldest deadline 2024-04-25T13:31:53.702256+0000) 2024-04-25T13:32:01.220 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:00 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:00.949+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-04-25T13:31:34.803260+0000 front 2024-04-25T13:31:34.803291+0000 (oldest deadline 2024-04-25T13:32:00.703288+0000) 2024-04-25T13:32:02.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:01 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:01.985+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-25T13:30:33.494282+0000 front 2024-04-25T13:30:33.494316+0000 (oldest deadline 2024-04-25T13:30:56.394345+0000) 2024-04-25T13:32:02.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:01 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:01.985+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6804 osd.1 since back 2024-04-25T13:30:41.095336+0000 front 2024-04-25T13:30:41.095315+0000 (oldest deadline 2024-04-25T13:31:06.995403+0000) 2024-04-25T13:32:02.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:01 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:01.985+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T13:30:50.496285+0000 front 2024-04-25T13:30:50.496229+0000 (oldest deadline 2024-04-25T13:31:15.796380+0000) 2024-04-25T13:32:02.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:01 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:01.985+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-04-25T13:30:58.697388+0000 front 2024-04-25T13:30:58.697548+0000 (oldest deadline 2024-04-25T13:31:19.197424+0000) 2024-04-25T13:32:02.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:01 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:01.985+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6812 osd.4 since back 2024-04-25T13:31:01.498732+0000 front 2024-04-25T13:31:01.498681+0000 (oldest deadline 2024-04-25T13:31:28.499335+0000) 2024-04-25T13:32:02.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:01 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:01.985+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T13:31:13.800250+0000 front 2024-04-25T13:31:13.800232+0000 (oldest deadline 2024-04-25T13:31:37.900250+0000) 2024-04-25T13:32:02.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:01 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:01.985+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.66:6820 osd.7 since back 2024-04-25T13:31:29.002634+0000 front 2024-04-25T13:31:29.002587+0000 (oldest deadline 2024-04-25T13:31:53.702256+0000) 2024-04-25T13:32:02.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:01 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-25T13:32:01.985+0000 7f96e86e1700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-04-25T13:31:34.803260+0000 front 2024-04-25T13:31:34.803291+0000 (oldest deadline 2024-04-25T13:32:00.703288+0000) 2024-04-25T13:32:03.083 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:02 smithi136 podman[69849]: 2024-04-25 13:32:02.809306301 +0000 UTC m=+5.538741175 container died 41aa8a6db1165d54cd91a02ede5134625bdf4c82818478ef41f30c290c78a9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:32:03.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:03 smithi136 podman[69849]: 2024-04-25 13:32:03.190429269 +0000 UTC m=+5.919864142 container cleanup 41aa8a6db1165d54cd91a02ede5134625bdf4c82818478ef41f30c290c78a9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T13:32:03.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:03 smithi136 bash[69849]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11 2024-04-25T13:32:03.784 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:03 smithi136 podman[69868]: 2024-04-25 13:32:03.483220413 +0000 UTC m=+0.664185200 container remove 41aa8a6db1165d54cd91a02ede5134625bdf4c82818478ef41f30c290c78a9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T13:32:04.095 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:03 smithi136 podman[69961]: 2024-04-25 13:32:03.687592658 +0000 UTC m=+0.017965109 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:32:04.095 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:03 smithi136 podman[69961]: 2024-04-25 13:32:03.792103449 +0000 UTC m=+0.122475887 container create 8a84029cc8076b7bc37b71235db3f5a2c42928dc409d861e76cdaf7156eb9638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T13:32:04.095 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:04 smithi136 podman[69961]: 2024-04-25 13:32:04.011147755 +0000 UTC m=+0.341520196 container init 8a84029cc8076b7bc37b71235db3f5a2c42928dc409d861e76cdaf7156eb9638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:32:04.095 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:04 smithi136 podman[69961]: 2024-04-25 13:32:04.014542417 +0000 UTC m=+0.344914857 container start 8a84029cc8076b7bc37b71235db3f5a2c42928dc409d861e76cdaf7156eb9638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:32:04.469 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:04 smithi136 podman[69961]: 2024-04-25 13:32:04.094447634 +0000 UTC m=+0.424820078 container attach 8a84029cc8076b7bc37b71235db3f5a2c42928dc409d861e76cdaf7156eb9638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-04-25T13:32:04.470 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 13:32:04 smithi136 podman[69961]: 2024-04-25 13:32:04.302175652 +0000 UTC m=+0.632548089 container died 8a84029cc8076b7bc37b71235db3f5a2c42928dc409d861e76cdaf7156eb9638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-04-25T13:32:04.875 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T13:32:04.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:04.914 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T13:32:04.914 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T13:32:04.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.a 2024-04-25T13:32:05.217 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:32:04 smithi031 systemd[1]: Stopping Ceph mds.a for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:05.484 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:32:05 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-a[71248]: 2024-04-25T13:32:05.216+0000 7f316c425700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:05.484 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:32:05 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-a[71248]: 2024-04-25T13:32:05.216+0000 7f316c425700 -1 mds.a *** got signal Terminated *** 2024-04-25T13:32:09.092 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:32:08 smithi031 podman[87677]: 2024-04-25 13:32:08.823090263 +0000 UTC m=+3.711320070 container died 8dcf8d9c8c9180f82af7af6c9e047aa3be7126e19ea08b792d7625d51f1a4ce1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T13:32:09.350 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:32:09 smithi031 podman[87677]: 2024-04-25 13:32:09.165728089 +0000 UTC m=+4.053957885 container cleanup 8dcf8d9c8c9180f82af7af6c9e047aa3be7126e19ea08b792d7625d51f1a4ce1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T13:32:09.350 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:32:09 smithi031 bash[87677]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-a 2024-04-25T13:32:09.605 INFO:journalctl@ceph.mds.a.smithi031.stdout:Apr 25 13:32:09 smithi031 podman[87690]: 2024-04-25 13:32:09.350084587 +0000 UTC m=+0.524747568 container remove 8dcf8d9c8c9180f82af7af6c9e047aa3be7126e19ea08b792d7625d51f1a4ce1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T13:32:09.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T13:32:09.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:09.645 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T13:32:09.645 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T13:32:09.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.d 2024-04-25T13:32:09.984 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:32:09 smithi031 systemd[1]: Stopping Ceph mds.d for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:10.484 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:32:09 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-d[70289]: 2024-04-25T13:32:09.986+0000 7f07f81c0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:10.484 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:32:09 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-d[70289]: 2024-04-25T13:32:09.986+0000 7f07f81c0700 -1 mds.d *** got signal Terminated *** 2024-04-25T13:32:13.637 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:32:13 smithi031 podman[87796]: 2024-04-25 13:32:13.365899463 +0000 UTC m=+3.491331140 container died 930113d5b4b64cd0d3398af044cf7e09090820dc4c210e335d8b7834fc55eb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T13:32:13.985 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:32:13 smithi031 podman[87796]: 2024-04-25 13:32:13.735597973 +0000 UTC m=+3.861029660 container cleanup 930113d5b4b64cd0d3398af044cf7e09090820dc4c210e335d8b7834fc55eb60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:32:13.985 INFO:journalctl@ceph.mds.d.smithi031.stdout:Apr 25 13:32:13 smithi031 bash[87796]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-d 2024-04-25T13:32:14.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T13:32:14.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:14.198 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T13:32:14.198 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T13:32:14.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.g 2024-04-25T13:32:14.514 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:32:14 smithi031 systemd[1]: Stopping Ceph mds.g for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:14.984 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:32:14 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-g[70899]: 2024-04-25T13:32:14.514+0000 7fa660e97700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:14.984 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:32:14 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-g[70899]: 2024-04-25T13:32:14.514+0000 7fa660e97700 -1 mds.g *** got signal Terminated *** 2024-04-25T13:32:18.323 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:32:18 smithi031 podman[87916]: 2024-04-25 13:32:18.057418671 +0000 UTC m=+3.630303533 container died f21d230cf9317af5bc8cfe8d0577c41c0c36c2f98aa3e7c375cbceebd3b773b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T13:32:18.715 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:32:18 smithi031 podman[87916]: 2024-04-25 13:32:18.422146495 +0000 UTC m=+3.995031347 container cleanup f21d230cf9317af5bc8cfe8d0577c41c0c36c2f98aa3e7c375cbceebd3b773b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T13:32:18.715 INFO:journalctl@ceph.mds.g.smithi031.stdout:Apr 25 13:32:18 smithi031 bash[87916]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-g 2024-04-25T13:32:18.854 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T13:32:18.886 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:18.886 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T13:32:18.887 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T13:32:18.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.j 2024-04-25T13:32:18.984 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:32:18 smithi031 systemd[1]: Stopping Ceph mds.j for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:19.484 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:32:19 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-j[69937]: 2024-04-25T13:32:19.200+0000 7f8db517e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:19.485 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:32:19 smithi031 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-j[69937]: 2024-04-25T13:32:19.200+0000 7f8db517e700 -1 mds.j *** got signal Terminated *** 2024-04-25T13:32:22.842 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:32:22 smithi031 podman[88038]: 2024-04-25 13:32:22.557949843 +0000 UTC m=+3.453343005 container died b246bde6ed8e866649658f1b34fa02d614b813e0f4612b7fb943a1cd020ec2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T13:32:23.167 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:32:22 smithi031 podman[88038]: 2024-04-25 13:32:22.933487546 +0000 UTC m=+3.828880704 container cleanup b246bde6ed8e866649658f1b34fa02d614b813e0f4612b7fb943a1cd020ec2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T13:32:23.168 INFO:journalctl@ceph.mds.j.smithi031.stdout:Apr 25 13:32:22 smithi031 bash[88038]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-j 2024-04-25T13:32:23.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T13:32:23.334 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:23.335 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T13:32:23.335 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T13:32:23.336 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.b 2024-04-25T13:32:23.644 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:32:23 smithi066 systemd[1]: Stopping Ceph mds.b for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:23.908 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:32:23 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-b[63639]: 2024-04-25T13:32:23.641+0000 7f5340205700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:23.908 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:32:23 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-b[63639]: 2024-04-25T13:32:23.641+0000 7f5340205700 -1 mds.b *** got signal Terminated *** 2024-04-25T13:32:27.513 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:32:27 smithi066 podman[70129]: 2024-04-25 13:32:27.24997947 +0000 UTC m=+3.714836463 container died 3ff34147d10f4cb786c0adb36a72c48969cf712304a9396ccec46d3bf60c874f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:32:27.801 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:32:27 smithi066 podman[70129]: 2024-04-25 13:32:27.533969612 +0000 UTC m=+3.998826613 container cleanup 3ff34147d10f4cb786c0adb36a72c48969cf712304a9396ccec46d3bf60c874f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T13:32:27.802 INFO:journalctl@ceph.mds.b.smithi066.stdout:Apr 25 13:32:27 smithi066 bash[70129]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-b 2024-04-25T13:32:27.934 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T13:32:27.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:27.970 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T13:32:27.970 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T13:32:27.970 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.e 2024-04-25T13:32:28.158 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:32:28 smithi066 systemd[1]: Stopping Ceph mds.e for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:28.658 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:32:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-e[63018]: 2024-04-25T13:32:28.295+0000 7f8f54346700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:28.658 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:32:28 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-e[63018]: 2024-04-25T13:32:28.295+0000 7f8f54346700 -1 mds.e *** got signal Terminated *** 2024-04-25T13:32:32.660 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:32:32 smithi066 podman[70250]: 2024-04-25 13:32:32.305965913 +0000 UTC m=+4.116300251 container died 549fb8454996fa3fb1da6bf28b3b9030d3577aee7f88ddb3963b428b467a9027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:32:33.158 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:32:32 smithi066 podman[70250]: 2024-04-25 13:32:32.887458734 +0000 UTC m=+4.697793066 container cleanup 549fb8454996fa3fb1da6bf28b3b9030d3577aee7f88ddb3963b428b467a9027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:32:33.159 INFO:journalctl@ceph.mds.e.smithi066.stdout:Apr 25 13:32:32 smithi066 bash[70250]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-e 2024-04-25T13:32:33.333 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T13:32:33.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:33.369 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T13:32:33.369 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T13:32:33.369 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.h 2024-04-25T13:32:33.708 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:32:33 smithi066 systemd[1]: Stopping Ceph mds.h for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:34.158 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:32:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-h[63331]: 2024-04-25T13:32:33.707+0000 7f745af1a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:34.159 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:32:33 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-h[63331]: 2024-04-25T13:32:33.707+0000 7f745af1a700 -1 mds.h *** got signal Terminated *** 2024-04-25T13:32:36.783 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:32:36 smithi066 podman[70384]: 2024-04-25 13:32:36.498302374 +0000 UTC m=+2.898073294 container died b9d5d6c3c08be5d4e0148deab4eeb4a102abc7186bfe1bbc8ec0ee62bbda3f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T13:32:37.117 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:32:36 smithi066 podman[70384]: 2024-04-25 13:32:36.882831964 +0000 UTC m=+3.282602881 container cleanup b9d5d6c3c08be5d4e0148deab4eeb4a102abc7186bfe1bbc8ec0ee62bbda3f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:32:37.117 INFO:journalctl@ceph.mds.h.smithi066.stdout:Apr 25 13:32:36 smithi066 bash[70384]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-h 2024-04-25T13:32:37.246 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T13:32:37.278 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:37.279 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T13:32:37.279 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T13:32:37.279 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.k 2024-04-25T13:32:37.408 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:32:37 smithi066 systemd[1]: Stopping Ceph mds.k for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:38.158 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:32:37 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-k[62699]: 2024-04-25T13:32:37.728+0000 7ff44f4b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:38.158 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:32:37 smithi066 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-k[62699]: 2024-04-25T13:32:37.728+0000 7ff44f4b4700 -1 mds.k *** got signal Terminated *** 2024-04-25T13:32:39.962 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:32:39 smithi066 podman[70504]: 2024-04-25 13:32:39.709183168 +0000 UTC m=+2.207517934 container died cb5e6fdeda399e3ff54e117dcebbe8c560d542b2a0b30c5ead968fef3e1f7da6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T13:32:40.350 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:32:40 smithi066 podman[70504]: 2024-04-25 13:32:40.161454434 +0000 UTC m=+2.659789201 container cleanup cb5e6fdeda399e3ff54e117dcebbe8c560d542b2a0b30c5ead968fef3e1f7da6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:32:40.351 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:32:40 smithi066 bash[70504]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-k 2024-04-25T13:32:40.602 INFO:journalctl@ceph.mds.k.smithi066.stdout:Apr 25 13:32:40 smithi066 podman[70518]: 2024-04-25 13:32:40.454381762 +0000 UTC m=+0.741786325 container remove cb5e6fdeda399e3ff54e117dcebbe8c560d542b2a0b30c5ead968fef3e1f7da6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T13:32:40.605 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T13:32:40.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:40.637 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T13:32:40.637 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T13:32:40.637 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.c 2024-04-25T13:32:40.969 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 13:32:40 smithi136 systemd[1]: Stopping Ceph mds.c for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:41.227 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 13:32:41 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-c[63340]: 2024-04-25T13:32:41.001+0000 7f9bb9a96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:41.227 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 13:32:41 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-c[63340]: 2024-04-25T13:32:41.001+0000 7f9bb9a96700 -1 mds.c *** got signal Terminated *** 2024-04-25T13:32:45.294 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 13:32:45 smithi136 podman[70065]: 2024-04-25 13:32:45.009187239 +0000 UTC m=+4.187817393 container died 3653f2f067337ae49a77ff761886ba82bfcf8cbfdd4fa8b32f6c23733e267089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:32:45.644 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 13:32:45 smithi136 podman[70065]: 2024-04-25 13:32:45.384836559 +0000 UTC m=+4.563466713 container cleanup 3653f2f067337ae49a77ff761886ba82bfcf8cbfdd4fa8b32f6c23733e267089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:32:45.645 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 13:32:45 smithi136 bash[70065]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-c 2024-04-25T13:32:45.771 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T13:32:45.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:45.806 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T13:32:45.807 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T13:32:45.807 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.f 2024-04-25T13:32:45.969 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:32:45 smithi136 systemd[1]: Stopping Ceph mds.f for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:46.469 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:32:46 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-f[63027]: 2024-04-25T13:32:46.196+0000 7fbeb0d19700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:46.469 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:32:46 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-f[63027]: 2024-04-25T13:32:46.197+0000 7fbeb0d19700 -1 mds.f *** got signal Terminated *** 2024-04-25T13:32:50.469 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:32:50 smithi136 podman[70201]: 2024-04-25 13:32:50.210602152 +0000 UTC m=+4.186338740 container died 2e9a08925b534a137041c6dba50e498b5f90288be736428dae62262f5cd87b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:32:50.940 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:32:50 smithi136 podman[70201]: 2024-04-25 13:32:50.672373656 +0000 UTC m=+4.648110241 container cleanup 2e9a08925b534a137041c6dba50e498b5f90288be736428dae62262f5cd87b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-f, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-25T13:32:50.940 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 13:32:50 smithi136 bash[70201]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-f 2024-04-25T13:32:51.090 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T13:32:51.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:51.121 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T13:32:51.121 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T13:32:51.121 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.i 2024-04-25T13:32:51.469 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:32:51 smithi136 systemd[1]: Stopping Ceph mds.i for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:51.469 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:32:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i[63643]: 2024-04-25T13:32:51.425+0000 7f917763d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:51.469 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:32:51 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i[63643]: 2024-04-25T13:32:51.425+0000 7f917763d700 -1 mds.i *** got signal Terminated *** 2024-04-25T13:32:55.518 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:32:55 smithi136 podman[70322]: 2024-04-25 13:32:55.244359692 +0000 UTC m=+3.906999875 container died 7ba737bdf5b1a6bdb9a14727625dfd8ec380af7ad373f193d399951c90e5ae4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T13:32:55.926 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:32:55 smithi136 podman[70322]: 2024-04-25 13:32:55.617499918 +0000 UTC m=+4.280140107 container cleanup 7ba737bdf5b1a6bdb9a14727625dfd8ec380af7ad373f193d399951c90e5ae4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T13:32:55.926 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 13:32:55 smithi136 bash[70322]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-i 2024-04-25T13:32:56.050 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T13:32:56.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:32:56.081 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T13:32:56.082 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T13:32:56.082 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.l 2024-04-25T13:32:56.219 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 13:32:56 smithi136 systemd[1]: Stopping Ceph mds.l for e4b023f4-0304-11ef-bc93-c7b262605968... 2024-04-25T13:32:56.719 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 13:32:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-l[62722]: 2024-04-25T13:32:56.395+0000 7f01cb80a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:32:56.719 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 13:32:56 smithi136 ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-l[62722]: 2024-04-25T13:32:56.395+0000 7f01cb80a700 -1 mds.l *** got signal Terminated *** 2024-04-25T13:33:00.719 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 13:33:00 smithi136 podman[70441]: 2024-04-25 13:33:00.401866216 +0000 UTC m=+4.104202060 container died dc2e90317663718d1b7d3ad47b0fc8b4b6b347af2ba87ddd94a49503b7a1a866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T13:33:01.071 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 13:33:00 smithi136 podman[70441]: 2024-04-25 13:33:00.820273045 +0000 UTC m=+4.522608895 container cleanup dc2e90317663718d1b7d3ad47b0fc8b4b6b347af2ba87ddd94a49503b7a1a866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T13:33:01.071 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 13:33:00 smithi136 bash[70441]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-l 2024-04-25T13:33:01.313 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T13:33:01.333 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 13:33:01 smithi136 podman[70455]: 2024-04-25 13:33:01.071165872 +0000 UTC m=+0.663524156 container remove dc2e90317663718d1b7d3ad47b0fc8b4b6b347af2ba87ddd94a49503b7a1a866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-e4b023f4-0304-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:33:01.334 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 13:33:01 smithi136 systemd[1]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.l.service: Deactivated successfully. 2024-04-25T13:33:01.334 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 13:33:01 smithi136 systemd[1]: Stopped Ceph mds.l for e4b023f4-0304-11ef-bc93-c7b262605968. 2024-04-25T13:33:01.334 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 13:33:01 smithi136 systemd[1]: ceph-e4b023f4-0304-11ef-bc93-c7b262605968@mds.l.service: Consumed 1.094s CPU time. 2024-04-25T13:33:01.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:33:01.344 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T13:33:01.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e4b023f4-0304-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:33:01.544 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:33:04.078 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e4b023f4-0304-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:33:04.279 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:33:06.763 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e4b023f4-0304-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:33:06.953 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:33:09.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:33:09.374 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:33:09.406 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:33:09.431 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T13:33:09.434 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465/remote/smithi031/crash 2024-04-25T13:33:09.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:33:09.475 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:33:09.475 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:33:09.477 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465/remote/smithi066/crash 2024-04-25T13:33:09.478 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:33:09.519 INFO:teuthology.orchestra.run.smithi066.stderr:tar: /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:33:09.519 INFO:teuthology.orchestra.run.smithi066.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:33:09.520 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465/remote/smithi136/crash 2024-04-25T13:33:09.521 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:33:09.562 INFO:teuthology.orchestra.run.smithi136.stderr:tar: /var/lib/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:33:09.562 INFO:teuthology.orchestra.run.smithi136.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:33:09.563 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T13:33:09.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T13:33:09.602 INFO:tasks.cephadm:Compressing logs... 2024-04-25T13:33:09.602 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:33:09.645 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:33:09.648 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:33:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:33:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:: No such file or directory 2024-04-25T13:33:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:33:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:33:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T13:33:09.688 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T13:33:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T13:33:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T13:33:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T13:33:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T13:33:09.689 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:33:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mgr.x.log: 90.4% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:33:09.689 INFO:teuthology.orchestra.run.smithi031.stderr: 94.7% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T13:33:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T13:33:09.689 INFO:teuthology.orchestra.run.smithi031.stderr: 94.6% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T13:33:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:33:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T13:33:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:33:09.690 INFO:teuthology.orchestra.run.smithi031.stderr: 84.2% 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:33:09.690 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:33:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T13:33:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T13:33:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T13:33:09.691 INFO:teuthology.orchestra.run.smithi136.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:33:09.692 INFO:teuthology.orchestra.run.smithi136.stderr:: No such file or directory 2024-04-25T13:33:09.692 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:33:09.692 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T13:33:09.692 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T13:33:09.692 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:33:09.692 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T13:33:09.692 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T13:33:09.692 INFO:teuthology.orchestra.run.smithi136.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:33:09.692 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T13:33:09.693 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T13:33:09.693 INFO:teuthology.orchestra.run.smithi136.stderr: 88.0% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.log.gz/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mon.c.log: 2024-04-25T13:33:09.693 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T13:33:09.693 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:33:09.693 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.i.log: /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.6% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:33:09.693 INFO:teuthology.orchestra.run.smithi136.stderr: 94.6% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T13:33:09.693 INFO:teuthology.orchestra.run.smithi136.stderr: 94.6% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T13:33:09.693 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T13:33:09.693 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T13:33:09.694 INFO:teuthology.orchestra.run.smithi136.stderr: 83.8%/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.11.log: -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:33:09.694 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:33:09.694 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T13:33:09.694 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.8.log: 91.9% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T13:33:09.694 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T13:33:09.695 INFO:teuthology.orchestra.run.smithi136.stderr: 91.0% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:33:09.696 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T13:33:09.699 INFO:teuthology.orchestra.run.smithi066.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-04-25T13:33:09.699 INFO:teuthology.orchestra.run.smithi066.stderr: -- /var/log/ceph/cephadm.log 2024-04-25T13:33:09.699 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:33:09.699 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:33:09.699 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T13:33:09.700 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T13:33:09.700 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.7.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:33:09.700 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T13:33:09.700 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.b.log: -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T13:33:09.700 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T13:33:09.700 INFO:teuthology.orchestra.run.smithi066.stderr: 88.0% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:33:09.700 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T13:33:09.700 INFO:teuthology.orchestra.run.smithi066.stderr:gzip/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mgr.y.log: -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T13:33:09.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:33:09.701 INFO:teuthology.orchestra.run.smithi066.stderr: 90.5% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:33:09.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T13:33:09.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:33:09.701 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:33:09.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T13:33:09.701 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T13:33:09.701 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-volume.log: 94.4% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T13:33:09.701 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T13:33:09.702 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.8% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T13:33:09.702 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T13:33:09.702 INFO:teuthology.orchestra.run.smithi066.stderr: 91.1% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:33:09.702 INFO:teuthology.orchestra.run.smithi031.stderr: 91.1% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:33:09.748 INFO:teuthology.orchestra.run.smithi066.stderr: 94.0% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T13:33:09.797 INFO:teuthology.orchestra.run.smithi136.stderr: 93.4% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T13:33:09.921 INFO:teuthology.orchestra.run.smithi031.stderr: 90.2% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T13:33:09.961 INFO:teuthology.orchestra.run.smithi066.stderr: 92.0% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T13:33:10.079 INFO:teuthology.orchestra.run.smithi136.stderr: 92.0% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T13:33:10.094 INFO:teuthology.orchestra.run.smithi136.stderr: 92.7% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T13:33:10.137 INFO:teuthology.orchestra.run.smithi066.stderr: 92.9% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T13:33:10.138 INFO:teuthology.orchestra.run.smithi136.stderr: 92.7% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T13:33:10.139 INFO:teuthology.orchestra.run.smithi031.stderr: 93.0% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T13:33:10.147 INFO:teuthology.orchestra.run.smithi066.stderr: 92.5% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T13:33:10.160 INFO:teuthology.orchestra.run.smithi136.stderr: 93.0% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T13:33:10.162 INFO:teuthology.orchestra.run.smithi031.stderr: 92.7% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T13:33:10.170 INFO:teuthology.orchestra.run.smithi031.stderr: 92.9% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T13:33:10.235 INFO:teuthology.orchestra.run.smithi066.stderr: 92.8% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T13:33:10.241 INFO:teuthology.orchestra.run.smithi136.stderr: 92.8% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T13:33:10.243 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-25T13:33:10.243 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m0.581s 2024-04-25T13:33:10.243 INFO:teuthology.orchestra.run.smithi136.stderr:user 0m2.405s 2024-04-25T13:33:10.243 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m0.121s 2024-04-25T13:33:10.266 INFO:teuthology.orchestra.run.smithi066.stderr: 92.9% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T13:33:10.268 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-04-25T13:33:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m0.606s 2024-04-25T13:33:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:user 0m2.408s 2024-04-25T13:33:10.269 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m0.099s 2024-04-25T13:33:10.358 INFO:teuthology.orchestra.run.smithi031.stderr: 92.8% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T13:33:10.606 INFO:teuthology.orchestra.run.smithi031.stderr: 90.5% -- replaced with /var/log/ceph/e4b023f4-0304-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T13:33:10.608 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-04-25T13:33:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m0.948s 2024-04-25T13:33:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m3.232s 2024-04-25T13:33:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m0.135s 2024-04-25T13:33:10.609 INFO:tasks.cephadm:Archiving logs... 2024-04-25T13:33:10.609 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465/remote/smithi031/log 2024-04-25T13:33:10.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:33:11.117 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465/remote/smithi066/log 2024-04-25T13:33:11.118 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:33:11.486 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465/remote/smithi136/log 2024-04-25T13:33:11.487 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:33:11.871 INFO:tasks.cephadm:Removing cluster... 2024-04-25T13:33:11.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e4b023f4-0304-11ef-bc93-c7b262605968 --force 2024-04-25T13:33:12.072 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:33:12.337 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e4b023f4-0304-11ef-bc93-c7b262605968 --force 2024-04-25T13:33:12.538 INFO:teuthology.orchestra.run.smithi066.stdout:Deleting cluster with fsid: e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:33:12.797 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e4b023f4-0304-11ef-bc93-c7b262605968 --force 2024-04-25T13:33:12.975 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: e4b023f4-0304-11ef-bc93-c7b262605968 2024-04-25T13:33:13.227 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T13:33:13.228 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:33:13.244 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:33:13.261 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:33:13.275 INFO:tasks.cephadm:Teardown complete 2024-04-25T13:33:13.276 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T13:33:13.288 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T13:33:13.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:33:13.290 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:33:13.303 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:33:13.413 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T13:33:13.426 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T13:33:13.426 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:33:13.429 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:33:13.432 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:33:13.441 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:33:13.444 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:33:13.446 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:33:13.446 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:33:13.446 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-04-25T13:33:13.446 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +935us[ +922us] +/- 62ms 2024-04-25T13:33:13.447 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -1628us[-1628us] +/- 70ms 2024-04-25T13:33:13.447 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +676us[ +676us] +/- 78ms 2024-04-25T13:33:13.447 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:33:13.449 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:33:13.449 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-04-25T13:33:13.449 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 +930us[ +953us] +/- 62ms 2024-04-25T13:33:13.450 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -1617us[-1594us] +/- 70ms 2024-04-25T13:33:13.450 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +667us[ +690us] +/- 78ms 2024-04-25T13:33:13.450 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:33:13.451 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:33:13.451 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-04-25T13:33:13.451 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +911us[ +894us] +/- 62ms 2024-04-25T13:33:13.451 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -1636us[-1636us] +/- 70ms 2024-04-25T13:33:13.451 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +648us[ +631us] +/- 78ms 2024-04-25T13:33:13.451 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:33:13.452 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T13:33:13.465 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T13:33:13.465 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T13:33:13.482 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:33:13.561 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:33:13.629 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:33:13.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:33:13.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:33:13.721 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:33:13.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:33:13.749 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:33:13.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:33:13.775 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T13:33:13.786 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T13:33:13.802 INFO:teuthology.task.internal:Duration was 1848.038138 seconds 2024-04-25T13:33:13.803 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T13:33:13.819 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T13:33:13.820 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:33:13.822 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:33:13.825 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:33:13.858 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:33:13.861 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:33:13.861 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:33:14.201 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T13:33:14.201 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-04-25T13:33:14.201 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:33:14.221 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-04-25T13:33:14.222 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:33:14.268 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-04-25T13:33:14.269 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:33:14.291 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T13:33:14.291 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:33:14.293 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:33:14.312 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:33:14.356 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T13:33:14.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:33:14.359 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:33:14.377 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:33:14.733 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T13:33:14.745 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T13:33:14.745 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:33:14.769 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:33:14.793 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:33:14.817 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T13:33:14.828 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:33:14.830 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:33:14.837 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:33:14.850 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-04-25T13:33:14.855 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-04-25T13:33:14.879 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-04-25T13:33:14.901 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:33:14.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:33:14.922 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:33:14.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:33:14.936 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:33:14.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:33:14.955 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T13:33:14.968 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T13:33:14.969 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465/remote/smithi031 2024-04-25T13:33:14.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:33:15.009 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465/remote/smithi066 2024-04-25T13:33:15.009 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:33:15.047 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465/remote/smithi136 2024-04-25T13:33:15.047 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:33:15.081 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T13:33:15.082 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:33:15.084 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:33:15.086 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:33:15.130 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T13:33:15.144 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T13:33:15.144 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T13:33:15.159 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T13:33:15.160 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:33:15.162 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:33:15.164 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:33:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:33 /home/ubuntu/cephtest 2024-04-25T13:33:15.177 INFO:teuthology.orchestra.run.smithi066.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:33 /home/ubuntu/cephtest 2024-04-25T13:33:15.184 INFO:teuthology.orchestra.run.smithi136.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:33 /home/ubuntu/cephtest 2024-04-25T13:33:15.185 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T13:33:15.201 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T13:33:15.236 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T13:33:15.261 DEBUG:teuthology.nuke:shortname: smithi031 2024-04-25T13:33:15.261 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:33:15.283 DEBUG:teuthology.nuke:shortname: smithi066 2024-04-25T13:33:15.283 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:33:15.289 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:55:27.200103', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBENxculZ/TityTJ5EObalVdLeOU1lw0PUsQ4uJQS0xZgVWQ/NbSSG2Unzb3VdtTAauyty/I7WY1BIxfTdzEtP/Q='} 2024-04-25T13:33:15.309 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:55:27.201363', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIgCIB34l20qAyUdpOx7bVIvJKOsg8rPP8WrxCRIbM+XOXl6uNMhgF5UR4qkSY12zGH1s5GKDR0Zyf5DgWuwHM='} 2024-04-25T13:33:15.314 DEBUG:teuthology.nuke:shortname: smithi136 2024-04-25T13:33:15.314 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:33:15.343 INFO:teuthology.orchestra.console.smithi031:Power off 2024-04-25T13:33:15.344 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:33:15.358 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673465', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 12:55:27.202430', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKJxYTDXhsmnnsvr4HCEHoYAQZDi3Gu03CCr10KeuA/HLJ9vpoMpowtuVVY3cC2gh06LA4fTkKS+v5EfIAK8XMM='} 2024-04-25T13:33:15.370 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2024-04-25T13:33:15.370 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:33:15.391 INFO:teuthology.orchestra.console.smithi066:Power off 2024-04-25T13:33:15.391 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:33:15.402 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-04-25T13:33:15.414 DEBUG:teuthology.orchestra.console.smithi066:power off output: Chassis Power Control: Down/Off 2024-04-25T13:33:15.415 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:33:15.430 INFO:teuthology.orchestra.console.smithi136:Power off 2024-04-25T13:33:15.430 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:33:15.440 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-04-25T13:33:15.455 DEBUG:teuthology.orchestra.console.smithi136:power off output: Chassis Power Control: Down/Off 2024-04-25T13:33:15.455 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:33:15.478 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-04-25T13:33:19.403 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:33:19.441 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:33:19.480 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:33:19.515 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-04-25T13:33:19.552 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-04-25T13:33:19.590 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-04-25T13:33:23.518 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:33:23.553 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:33:23.590 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:33:23.629 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2024-04-25T13:33:23.666 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on 2024-04-25T13:33:23.702 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-04-25T13:33:23.730 INFO:teuthology.orchestra.console.smithi031:Power off completed 2024-04-25T13:33:27.670 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:33:27.703 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:33:27.784 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off 2024-04-25T13:33:27.813 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2024-04-25T13:33:27.885 INFO:teuthology.orchestra.console.smithi066:Power off completed 2024-04-25T13:33:27.914 INFO:teuthology.orchestra.console.smithi136:Power off completed 2024-04-25T13:33:28.020 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} duration: 1848.0381381511688 failure_reason: '''NoneType'' object has no attribute ''get''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=460927445d5d47519d04b22f4c6cf1fd status: fail success: false 2024-04-25T13:33:28.021 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:33:28.098 INFO:teuthology.run:FAIL